false positive minimization methods and apparatus in face recognition applications

专利摘要:
The present invention relates to an apparatus which may include a processor that may receive location data from a user device, and store location data in a user profile data structure also storing face recognition data. The processor may also receive at least one image, and may identify a location based at least in part on a feature set within the at least one image. The processor may, for each user profile data structure stored in a database, compare the location data in that user profile data structure with the location. The processor may, when the user profile data structure location data and location match, conduct a face recognition to determine whether the user associated with the user profile data structure can be identified in at least one image. The processor can then associate at least one image with the user profile data structure if the user can be identified.
公开号:BR112018007979A2
申请号:R112018007979
申请日:2016-10-21
公开日:2019-11-12
发明作者:Resnick Adam；Joshpe Brett；Sabitov Ruslan
申请人:15 Seconds Of Fame Inc；
IPC主号:

专利说明:

Descriptive Report of the Invention Patent for METHODS AND APPARATUS FOR MINIMIZATION OF FALSE POSITIVE IN
FACIAL RECOGNITION APPLICATIONS.
CROSS REFERENCE TO RELATIVE APPLICATION [001] This application claims priority for and the benefit of US Provisional Application Serial Number 62 / 244,419, filed on October 21, 2015, and entitled METHODS AND APPARATUS FOR FALSE POSITIVE MINIMIZATION IN FACIAL RECOGNITION APPLICATIONS . The entire content of the application mentioned above is hereby expressly incorporated by reference.
BACKGROUND [002] The modalities described here generally refer to facial recognition and video analytics, and more specifically, to apparatus and methods for minimizing false positives in facial recognition applications.
[003] Increases in the availability and capacity of electronic devices such as cameras, tablets, smartphones, etc. allowed some people to take pictures and / or capture video of their experiences. For example, the inclusion and improvement of cameras on smartphones, tablets, and / or other similar devices has led to an increase in these devices being used to make images (for example, photographic data, image data, etc.) and videos (for example , video stream data). Although, it has become easier for some people to make images and / or videos of their experiences, in some cases, there may still be challenges in including the desired parts (including the person who would otherwise be making the image or video) . Furthermore, a person usually needs to remember and / or have a chance to make the image and / or video, and failing to do so can result in a missed opportunity.
[004] In some cases, places and / or events such as events
Petition 870180048519, of 06/07/2018, p. 5/90
2/68 sports, concerts, rallies, graduation, and / or the like have cameras that can take images and / or video of those in attendance. In some cases, however, analyzing, parsing, and / or otherwise making images and / or video streams available may use a relatively large amount of resources, may be inaccurate, and / or may fail to provide contextual data associated or similar. More specifically, in some cases, it can be difficult to verify that a specific person detected in an image, was actually in the location captured in the image, due to false positives obtained from using facial recognition only to identify people in images.
[005] Thus, a need exists for an improved device and methods to use contextual and location data to minimize false positives in, for example, public events.
SUMMARY [006] In some implementations, a device may include a memory and a processor operatively coupled to the memory. The processor can, in the first instance, receive location data from a user device, and can store the location data in a user profile data structure. The user profile data structure can include a user's face recognition data from the user device associated with the user based on at least one of two-dimensional facial recognition analytics, three-dimensional facial recognition analytics, or convolutional neural networks (CNN) . The processor can receive, in a second time different from the first time, at least one image from an image capture device. The processor can identify a location based at least in part on a set of characteristics with at least one image received, and can retrieve multiple data structures from a database
Petition 870180048519, of 06/07/2018, p. 6/90
3/68 user profile including the user profile data structure. The processor can, for each user profile data structure of multiple user profile data structures, compare the location data in that user profile data structure with the location. The processor can, when the location data of the user profile data structure and the location are within a predetermined distance from each other, determine whether the user associated with the user profile data structure can be identified in at least one Image. For example, the processor can analyze at least one image for the user's facial recognition data based on at least one of the two-dimensional facial recognition analytics, three-dimensional facial recognition analytics, or CNN to identify a confidence score facial recognition. The processor can then associate at least one image with the user profile data structure based on the facial recognition confidence score meeting a predetermined criterion.
BRIEF DESCRIPTION OF THE DRAWINGS [007] Figure 1A is a schematic illustration of a recognition system according to a modality.
[008] Figure 1B is a schematic illustration of a recognition system according to another modality.
[009] Figure 2 is a schematic illustration of a host device included in the recognition system in Figure 1.
[0010] Figure 3 is a flow chart illustrating a method of using a video recognition system according to a modality.
[0011] Figure 4 is a flow chart illustrating a method of using a video recognition system according to another modality.
Petition 870180048519, of 06/07/2018, p. 7/90
4/68 [0012] Figure 5 is an illustration of an image capture device capturing contextual information on media, according to a modality.
[0013] Figure 6 is a logical flowchart that illustrates using contextual information on media, and location data, to identify a user on the media, according to a modality.
DETAILED DESCRIPTION [0014] In some implementations, a device may include a memory and a processor operatively coupled to the memory. The processor can, in the first instance, receive location data from a user device, and can store the location data in a user profile data structure. The user profile data structure can include a user's face recognition data from the user device associated with the user based on at least one of two-dimensional facial recognition analytics, three-dimensional facial recognition analytics, or convolutional neural networks (CNN) . The processor can receive, in a second time different from the first time, at least one image from an image capture device. The processor can identify a location based at least in part on a set of characteristics with at least one image received, and can retrieve from a database multiple user profile data structures that include the user data structure. user profile. The processor can, for each user profile data structure of the multiple user profile data structures, compare the location data in that user profile data structure with the location. The processor can, when the location data of the user profile data structure and the location are within a predetermined distance from each other, determine whether the user associated with the user profile data structure
Petition 870180048519, of 06/07/2018, p. 8/90
5/68 can be identified in at least one image. For example, the processor can analyze at least one image for the user's facial recognition data based on at least one of the two-dimensional facial recognition analytics, three-dimensional facial recognition analytics, or CNN to identify a confidence score facial recognition. The processor can then associate at least one image with the user profile data structure based on the facial recognition confidence score meeting a predetermined criterion.
[0015] The modalities described here refer to detecting a user in the media based on facial recognition data and location information. In some embodiments, an image analysis method includes receiving, on a host device and a client device over a network, a signal indicating user check-ins at a location. The user can check in via his mobile device. An image capture device can capture media (for example, photos, videos, audio, and / or similar content) that can include the user. The host device can use the panorama and / or other background information on the media (for example, after processing the media using image processing techniques) to determine a specific location where the media was captured. The host device can also receive location information for the image capture device, for example, to verify the location of the image capture device and / or the location that the media detects. The host device can match the location detected on the media, with location data from users who have checked in, to determine which users have checked in at a location close to where the media was captured. The host device can then run
Petition 870180048519, of 06/07/2018, p. 9/90
6/68 perform image processing on the media to determine whether users who have checked in close to the location on the media appear on the media. The host device can send notifications to users that the host device detects on the media. In this mode, the host device can reduce the number of users to search for a specific media file, and reduce false positives by tying both the user's location and the user's appearance to the data obtained from the media.
[0016] As used in this specification, singular forms one, one and o include plural referents unless the context clearly dictates otherwise. Thus, for example, the term a module is intended to mean a single module or a combination of modules, a network is intended to mean one or more networks, or a combination thereof.
[0017] As used here the term module refers to any assembly and / or set of operatively coupled electrical components that may include, for example, a memory, a processor, electrical tracks, optical connectors, software (running on hardware), and / or the like. For example, a module running on the processor can be any combination of hardware-based modules (for example, a network of field programmable ports (FPGA), an application-specific integrated circuit (ASIC), a digital signal processor (DSP) ) and / or software-based module (for example, a computer code module stored in memory and / or executed on the processor) capable of performing one or more specific functions associated with that module.
[0018] The modalities and methods described here can use facial recognition data to (1) search for one or more images of a registered user (for example, a person whose facial recognition data is predetermined) in a flow of
Petition 870180048519, of 06/07/2018, p. 10/90
7/68 video and (2) provide a video stream that includes contextual data for a customer device associated with the user (for example, a smartphone, tablet, computer, usable electronic device, etc.). Facial recognition usually involves analyzing one or more images of a person's face to determine, for example, salient features of their facial structure (for example, cheekbones, chin, ears, eyes, jaw, nose, hairline, etc. ) and then define a set of qualitative and / or quantitative data associated with and / or otherwise representing the salient features. A proposal, for example, includes extracting data associated with the salient features of a person's face and defining a data set that includes geometric and / or coordinate-based information (for example, a three-dimensional (3D) analysis of facial recognition data) . Another proposal, for example, includes distilling image data into qualitative values and comparing these values with templates or the like (for example, a two-dimensional (2D) analysis of facial recognition data). In some cases, another proposal may include any suitable combination of 3D and 2D analytics.
[0019] Some facial recognition methods and / or algorithms include Principal Component Analysis using Eigenfaces (for example, Eigenvector associated with facial recognition), Linear Discriminative Analysis, Elastic Bunch Graph Match using the Fisherface algorithm, Hidden Markov model, Multilinear Subspace Learning that uses tensor representation, neuronal motivated dynamic connection coincidence, convolutional neural networks (CNN), and / or the like or their combination. Any of the modalities and / or methods described herein can use and / or implement any method and / or algorithm of facial recognition or its combination such as those described above.
[0020] Figure 1A is a schematic illustration of a system
Petition 870180048519, of 06/07/2018, p. 11/90
8/68 video recognition 100 according to a modality. In some cases, the video recognition system 100 (also referred to herein as the system) can be used to present a user's video stream based at least in part on facial recognition data. At least a portion of system 100 can be represented and / or described, for example, by a set of instructions or code stored in memory and executed in a processor of an electronic device (for example, a host device, a server or group of devices). servers, a personal computer (PC), a network device, etc.) and / or the like. For example, in some embodiments, a host device can receive a signal associated with a request to record face recognition data associated with a user and in response, it can store face recognition data in a database. Similarly, the host device can receive a signal associated with the video stream data. In some cases, one or more processors on the host device may then execute a set of instructions or code, stored in a memory on the host device, associated with the analysis of video stream data to determine whether one or more user images are present on the video stream based at least in part on facial recognition data and / or location information (such as landmark data). If images are found in the video stream data, the one or more processors can isolate a portion associated with the video stream data. Furthermore, one or more processors can execute a set of instructions or code to (1) associate contextual data such as time, location, event, etc. with video stream data and (2) define a user contextual video stream. The one or more processors can then send, to a client device associated with the user, a signal indicating a
Petition 870180048519, of 06/07/2018, p. 12/90
9/68 instruction to present the user's contextual video stream on a client device display (for example, graphically rendering the contextual video stream on an instantiated interface on the client device).
[0021] System 100 includes a host device 110 in communication with a database 140, a client device 150, and an image capture system 160. Host device 110 can be any suitable host device such as a server or group of servers, a network management device, a personal computer (PC), a processing unit, and / or the like in electronic communication with database 140, client device 150, and the image capture system 160. For example, in this embodiment, host device 110 may be a server or group of servers (arranged in substantially the same location and / or facility or distributed in more than one location) in electronic communication such as database 140, the client device 150, and the image capture system 160 over a network 105, as described in more detail herein.
[0022] The client device 150 can be any suitable device such as a PC, a laptop, a convertible laptop, a tablet, a personal digital assistant (PDA), a smartphone, a wearable electronic device (for example, a smart watch , etc.), and / or the like. Although not shown in Figure 1, in some embodiments, the client device 150 may be an electronic device that includes at least one memory, a processor, a communication interface, a display, and one or more inputs. The memory, processor, communication interface, display, and input (s) can be connected and / or electrically coupled to each other in order to allow signals to be sent between them. For example 870180048519, from 06/07/2018, p. 13/90
10/68 pio, in some modalities, the memory can be a random access memory (RAM), a temporary memory, a hard drive, a read-only memory (ROM), an erasable programmable read-only memory (EPROM), and / or the like. The processor can be any suitable processing device configured to operate or execute a set of instructions or code (for example, stored in memory) such as a general purpose processor (GPP), a central processing unit (CPU), a unit accelerated processing unit (APU), a graphics processor unit (GPU), an Application Specific Integrated Circuit (ASIC), and / or the like. Such a processor can operate or execute a set of instructions or code stored in memory associated with the use of a PC application, a mobile application, an internet web browser, cellular and / or wireless communication (over a network), and / or the like. More specifically, the processor may execute a set of instructions or code stored in memory associated with sending facial recognition to and / or receiving facial recognition data and / or contextual video stream data from the host device 110, as described hereinafter. Details.
The communication interface of the client device 150 can be any suitable module and / or device that can put the resource in communication with the host device 110 such as one or more network interface cards or the like. Such a network interface card may include, for example, an Ethernet port, a WiFi® radio, a Bluetooth® radio, a near field communication radio (NFC), and / or a cellular radio that can place the client device 150 in communication with the host device 110 over a network (e.g., network 105) or the like. As such, the communication interface can send signals to and / or receive signals from the
Petition 870180048519, of 06/07/2018, p. 14/90
11/68 processor associated with electronic communication with host device 110 over network 105.
[0024] The display of the client device 150 can be, for example, a cathode ray tube (CRT) monitor, a liquid crystal display (LCD) monitor, a light emitting diode (LED) monitor, and / or similar that can graphically represent any suitable portion of system 100 (for example, a graphical user interface (GUI) associated with a webpage, PC application, mobile application, and / or the like). In some embodiments, such a display may be and / or may include a touch key configured to receive haptic user input. In some cases, the display may be configured to graphically represent the data associated with a facial recognition process and / or data associated with a video stream, as described in more detail here.
[0025] The input (s) of the client device 150 can be any suitable module and / or device that can receive one or more inputs (for example, user inputs) and that can send signals to and / or receive signals processor associated with one or more inputs. In some embodiments, the input (s) may be and / or may include ports, plugs, and / or other interfaces configured to be placed in electronic communication with a device. For example, such an input may be a universal serial bus (USB) port, an Institute of Electrical and Electronics Engineers (IEEE) 1394 (FireWire) port, a Thunderbolt port, a Lightning port, and / or the like. In some embodiments, the display may be included on a touch screen or the like configured to receive haptic user input.
[0026] In some modalities, an entrance can be a camera and / or other image formation device. For example, in some modalities, such a camera may be integrated into the device
Petition 870180048519, of 06/07/2018, p. 15/90
12/68 of client 150 (for example, as on smartphones, tablets, laptops, etc.) and / or can be in communication with the client device 150 through a port or similar (for example, such as those described above) . The camera can be any suitable image-forming device such as, for example, a webcam or a face-facing camera included in a smartphone or tablet (for example, a camera pointed in substantially the same direction as the display). In this mode, the user can manipulate the client device 150 to cause the camera to capture an image (for example, a photo) or a video. Furthermore, in some cases, the display may be configured to graphically render data associated with an image captured by the camera. For example, in some embodiments, the client device 150 may be a smartphone, tablet, or wearable electronic device that includes a forward facing camera. In some cases, the user can manipulate the client device 150 to make an image or video of himself or herself through the camera (for example, also known as a selfie).
[0027] In some cases, a camera (for example, an input) included in the client device 150 can be used to capture an image of the user's face, which in turn can be used to record associated facial recognition data to the user. Specifically, the user can manipulate the client device 150 so that the camera captures an image of the user's face. In some cases, the display may be configured to graphically render an indication, frame, boundary, guide, and / or any other suitable graphic representation of data, those who wanted to provide an indication to a user associated with a desired alignment for the image of the user's face. Once the camera captures the desired image, the processor can receive and / or retrieve associated data
Petition 870180048519, of 06/07/2018, p. 16/90
13/68 to the user's face image and, in turn, can execute a set of instructions or code (for example, stored in memory) associated with at least a portion of a facial recognition process. For example, in some cases, the processor may execute a set of instructions or code associated with verifying an alignment between an indication, frame, limit, etc. graphically rendered on the display and the captured image of the user's face. In some cases, client device 150 may be configured to send, over network 105, a signal associated with data representing the user's image to host device 110 when alignment is verified, and in response, host device 110 can perform any appropriate facial recognition process or processes on the data, as described in more detail here.
[0028] The image capture system 160 can be and / or can include any suitable device or devices configured to capture image data. For example, the image capture system 160 can be and / or can include one or more cameras and / or image recording devices configured to capture an image (e.g., a photo) and / or record a video stream. In some embodiments, the image capture system 160 may include multiple cameras in communication with a central computing device such as a server, a personal computer, a data storage device (for example, a network attached storage device ( NAS), a database, etc.), and / or the like. In such modalities, cameras can be autonomous (for example, they can capture image data without stimulation and / or user input), and they can each send image data to the central computing device (for example, over a connection wired or wireless, a port, a serial bus,
Petition 870180048519, of 06/07/2018, p. 17/90
14/68 a network, and / or the like), which in turn, can store the image data in a memory and / or other data storage device. Furthermore, the central computing device may be in communication with the host device 110 (for example, over network 105) and may be configured to send at least a portion of the image data to the host device 110. Although shown in Figure 1 as being in communication with the host device 110 over the network 105, in other embodiments, such a central computing device may be included in, a part of, and / or otherwise coupled to the host device 110. In still other embodiments , the cameras may be in communication with the host device 110 (for example, over the network 105) without such a central computing device.
[0029] In some modalities, the image capture system 160 may be associated with and / or owned by a location or similar such as, for example, a sports arena, a theme park, a theater, and / or any other location appropriate. In other embodiments, the image capture system 160 may be used within or at a location but owned by a different entity (for example, a licensed and / or otherwise authorized entity to use the image capture system 160 within or on-site such as, for example, a television camera at a sporting event). In still other embodiments, the image capture system 160 can include any number of client devices (e.g., user devices) or the like such as smartphones, tablets, etc., which can be used as cameras or recorders. In such embodiments, at least some of the client devices may be in communication with the host device 110 and / or a central computing device associated with the site (for example, as described above).
Petition 870180048519, of 06/07/2018, p. 18/90
15/68 [0030] For example, in some embodiments, the camera integrated into the client device 150 can form and / or comprise at least a portion of the image capture system 160, as shown in Figure 1B. In this mode, the user can manipulate the client device 150 to capture an image and / or video recording and in response, the client device 150 can upload and / or otherwise send the image (e.g., image data, photographic data, etc.) and / or video recording data for the host device 110. In some cases, the image and / or video recording data can be stored on the client device 150 for any suitable time and loaded and / or sent to host device 110 at a later time. Furthermore, the image and / or video recording data can be stored on the client device 150 after the image and / or video recording data is sent to the host device 110. That is, sending the image and / or video recording data does not erase and / or remove the image and / or video recording data from the client device 150 (for example, a copy of the data is sent to the host device 110). Thus, as shown in Figure 1B, the image capture system 160 does not need to be associated with a specific event and / or location. In such cases, the user can manipulate the client device 150 (for example, an application of the client device 150) to capture user-generated content (for example, images, image data, photographic data, video stream data , etc.) through the camera and / or recording device (for example, the image capture system 160) integrated in the client device 150.
[0031] In some cases, the image capture system 160 is configured to capture image data associated with a location and / or event. In other words, the image capture system
Petition 870180048519, of 06/07/2018, p. 19/90
16/68
160 is configured to capture image data within a predetermined, known, and / or given context. For example, in some cases, the image capture system 160 may include one or more image capture devices (for example, cameras and / or video recorders) that are installed in an arena or similar and that are configured to capture data images associated with patrons, guests, interpreters, etc. in the arena. In this mode, the image capture system 160 is configured to capture image data within the context of the arena and / or an event that occurs in the arena. Thus, the captured image data can be, for example, contextual image data. That is, the image data is associated with contextual data. As described in more detail here, host device 110 can receive image data and / or video stream data from image capture system 160 and data associated with the context (for example, contextual data associated with the arena and / or the event occurring in the arena, and / or any other contextual and / or suitable metadata) from any suitable and / or similar data source; you can associate contextual data with, for example, image data; can define a user-specific contextual image and / or user-specific contextual video stream associated with, for example, a client device user 150; and can send the user-specific contextual image and / or user-specific contextual video stream associated with the user to the client device 150.
[0032] As described above, the client device 150 and the image capture system 160 can be in communication with the host device 110 through one or more networks. For example, as shown in Figure 1A, the client device 150 and the image capture system 160 can be in communication with the host device 110 through its communication interface and
Petition 870180048519, of 06/07/2018, p. 20/90
17/68 the network 105. The network 105 can be any type of network such as, for example, a local area network (LAN), a virtual network as well as a virtual local area network (VLAN), an area network broadband (WAN), a metropolitan area network (MAN), worldwide interoperability for microwave access network (WiMAX), a cellular network, the Internet, and / or any other suitable network implemented as a wired network and / or wireless. For example, network 105 can be implemented as a wireless local area network (WLAN) based on the standards of the Institute of Electrical and Electronics Engineers (IEEE)
802.11 (also known as WiFi ®). Furthermore, network 105 may include a combination of networks of any type, such as, for example, a LAN or WLAN and the Internet. In some embodiments, client device 150 may communicate with host device 110 and network 105 via intermediate networks and / or alternate networks (not shown), which may be similar to or different from network 105. As such, the client device 150 can send data to and / or receive data from host device 110 using multiple communication modes (for example, associated with any of the networks described above) that may or may not be transmitted to host device 110 using a common network . For example, the client device 150 may be a mobile phone (for example, smartphone) connected to the host device 110 via a cellular network and the Internet (for example, network 105).
[0033] In some cases, the network may facilitate, for example, a pair of network section or similar. In such cases, the paired network section may include, for example, client devices and / or any other suitable electronic device, each of which shares a common characteristic. For example, in some cases, the peer network section may include any other suitable client devices (for example, an electronic device registered with the bank).
Petition 870180048519, of 06/07/2018, p. 21/90
18/68 data 140 and / or similar) that is within a predetermined proximity to a place, event, location, etc. For example, in some cases, such a paired network section may include any number of registered client devices present at a location (for example, a sport event). In some cases, the peer network section can be automatically established based on contextual data associated with the user and / or the client device. In other cases, the peer network section can be automatically established based on one or more users checking in and / or otherwise publishing their presence on site or similar (for example, shouting out the user's presence). In some cases, the user can check in at a time when the user arrived at an event or similar (for example, sport event, concert, wedding, birthday party, meeting, etc.), at a registration time , at a time to capture an image or video stream, and / or the like. Also, check-in can include identifying information such as, for example, geolocation data, date and time data, personal or user identification data, etc. In some implementations, a user can also, through an application on his client device 150, search for events and / or locations for which the contextual video stream data has been captured. The user can check in the event and / or locations that are returned from the survey. As described here, checking in to an event and / or location can initiate processing of the contextual video stream data associated with that event and / or location, for example, to determine whether or not the user can be matched with the data of contextual video streams.
[0034] In other cases, a user can manually establish a peer network section that includes, for example, a predetermined set or group of users. In some cases, such sections of
Petition 870180048519, of 06/07/2018, p. 22/90
19/68 peer networks may be public networks, private networks, and / or otherwise limited access networks. For example, in some cases, a user may request to join a network section and / or may receive an invitation to join a network section and / or the like. In some cases, establishing a peer network section can, for example, facilitate communication (for example, group chat sections or the like) and / or image and / or video data sharing between users included in the section network connection.
[0035] Host device 110 can be any suitable device configured to send data to and / or receive data from database 140, client device 150, and / or image capture system 160. In some embodiments, the host device 110 can function as, for example, a server device (for example, a web server device), a network management device, an administrator device, and / or so on. In some embodiments, the host device 110 may be a group of servers or devices housed together within or on the same board, cabinet, and / or facility or distributed on or over multiple boards, cabinets, and / or facilities. The host device 110 includes at least one memory 115, a processor 120, and a communication interface 125 (see, for example, Figure 2). In some embodiments, memory 115, processor 120, and communication interface 125 are connected and / or electrically coupled so that signals can be sent between memory 115, processor 120, and communication interface 125. The device host 110 may also include and / or may otherwise be operatively coupled to database 140 configured to store user data, facial recognition data, contextual data (e.g., associated with a time, location, location, event, etc. .), video streams, and / or the like.
Petition 870180048519, of 06/07/2018, p. 23/90
20/68 [0036] Memory 115 can be, for example, a RAM, a temporary memory, a hard drive, a database, a ROM, an EPROM, an EEPROM, and / or so on. In some cases, the memory 115 of the host device 110 includes a set of instructions or code used to perform one or more facial recognition actions and / or used to communicate (for example, send and / or receive) data with at least one device (e.g., client device 150) using one or more suitable communication modes. Processor 120 can be any suitable processor such as, for example, a GPP, a CPU, an APU, a GPU, a network processor, an initial interface processor, an ASIC, an FPGA, and / or the like. Thus, processor 120 may be configured to operate and / or execute a set of instructions, modules, and / or code stored in memory 115. For example, processor 120 may be configured to execute a set of instructions and / or associated modules a, inter alia, receive facial recognition data (for example, from client device 150), analyze facial recognition data, record and / or store facial recognition data, receive video stream data (for example, from image capture system 160), analyze video stream data and compare video stream data with facial recognition data, send video stream data (for example, to client device 150), receive and / or analyze characteristics of the video stream data (for example, location information determined based on such background milestone and / or background data included in the video stream data, and / or itself pillars), and / or any other suitable process, as further described herein. The communication interface 125 can be any suitable device that can put the host device 110 in communication with the database 140, the device
Petition 870180048519, of 06/07/2018, p. 24/90
21/68 client site 150, image capture device 160 and / or any other suitable device and / or service communicating with network 105 (for example, any device configured to gather and / or at least temporarily store such data such as facial recognition data, video streams, and / or the like). In some embodiments, the communication interface 125 may include one or more wired and / or wireless interfaces, such as, for example, network interface cards (NIC), Ethernet interfaces, optical carrier interfaces (OC), interfaces asynchronous transfer mode (ATM), interfaces and / or wireless (for example, a WiFi® radio, a Bluetooth® radio, an NFC radio, and / or the like).
[0037] Returning to Figure 1 A, the database 140 associated with the host device 110 can be any suitable database such as, for example, a relational database, an object database, a relational database object database, hierarchical database, network database, entity relationship database, structured query language (SQL) database, extensible markup language (XML) database, repository digital, media library, server or cloud storage, and / or the like. In some embodiments, host device 110 may be communicating with database 140 over any suitable network (for example, network 105) through communication interface 125. In such embodiments, database 140 may be included or stored by a network attached storage device (NAS) that can communicate with the host device 110 over network 105 and / or any other network (s). In other embodiments, the database may be stored in memory 115 of the host device 110. In still other embodiments, the database may be operatively coupled to the host device 110 via
Petition 870180048519, of 06/07/2018, p. 25/90
22/68 a cable, a bus, a server cabinet, and / or the like.
[0038] Database 140 may store and / or at least temporarily retain data associated with video recognition system 100. For example, in some cases, database 140 may store data associated with and / or otherwise representing user profiles, resource lists, facial recognition data, modes, and / or methods, contextual data (for example, associated with a time, location, location, event, etc.), video streams or their portions, information of location (such as milestone data), and / or the like. In other words, database 140 can store data associated with users whose facial image data has been registered by system 100 (for example, registered users). In some embodiments, database 140 may be and / or may include a relational database, in which data may be stored, for example, in tables, matrices, vectors, etc. according to the relational model. For example, in some cases, the host device 110 may be configured to store in the database 140 the video stream data received from a video or image source (for example, the image capture system 160) and contextual data associated with video stream data. In some cases, the video stream data and the contextual data associated with it may collectively define a contextual or similar video stream, as described in more detail here. In other cases, the video stream data may be stored in database 140 to be contextual or similar data.
[0039] In some implementations, user profiles can be user profile data structures that include information relating to users accessing the video stream data. For example, a user profile data structure may include a user profile identifier, face recognition data (for example,
Petition 870180048519, of 06/07/2018, p. 26/90
23/68 example, data obtained from a user image (for example, facial feature data) that can be used to match the user with an image of contextual video stream data), a list of identifiers associated with data structures of contextual video streams stored in database 140 and associated with the user, a list of identifiers associated with the user profile data structures of other users with which the user is associated (for example, as a friend and / or contact ), user location data, and / or the like.
[0040] In some implementations, users can add each other as friends within an application through which they access contextual video stream data. Users can also be automatically associated with each other, for example, when a user associated with a first user profile is a contact for another user associated with a second user profile. For example, a user who operates a client device may have a list of contacts, and / or other contact information, stored on the client device. The application can retrieve and import contact information, it can match contact information with information in at least one user profile in the database, and it can automatically associate this at least one user profile with this user. In some implementations, users can be associated with each other by storing a list of friends and / or contacts (for example, a list of user profile identifiers to be added as friends of a specific user) within each user profile of each user. When a user adds a friend and / or contact, the user can automatically be notified when the friend and / or contact records and / or receives contextual, and / or similar video stream data. In some implementations, host device 110 can also use
Petition 870180048519, of 06/07/2018, p. 27/90
24/68 stored between users to automatically process the contextual video stream data associated with the user (for example, to determine whether the user's friends and / or contacts can be found within the contextual video stream data). For example, when contextual video stream data is received, when a friend and / or contact is associated with the user, and / or the like, the host device 110 can automatically process the contextual video stream data to determine whether the data facial image associated with the user's friends and / or contacts can be matched with the contextual video stream data.
[0041] Although host device 110 is shown and described with reference to Figure 1 as including and / or otherwise being operatively coupled to database 140 (e.g., a single database), in some embodiments, the device host 110 can be operatively coupled to any database number. Such databases may be configured to store at least a portion of a data set associated with system 100. For example, in some embodiments, host device 110 may be operatively coupled to and / or otherwise communicating with a first bank of data configured to receive and at least temporarily store user data, user profiles, and / or the like and a second database configured to receive and at least temporarily store video stream data and contextual data associated with the stream data of video. In some embodiments, the host device 110 may be operatively coupled to and / or otherwise in communication with a database that is stored within or on the client device 150 and / or the image capture system 160. In other words , at least a portion of a database may be implemented in and / or stored
Petition 870180048519, of 06/07/2018, p. 28/90
25/68 by the client device 150 and / or the image capture system 160. In this mode, the host device 110 and, in some cases, database 140 can be in communication with any database number that can be physically arranged in a different location than the host device 110, despite being in communication with the host device 110 (for example, over network 105).
[0042] In some modalities, database 140 can be a searchable database and / or repository. For example, in some cases, database 140 can store video stream data associated with a user (for example, contextual video stream data). In some cases, the user may search database 140 to retrieve and / or view one or more contextual video streams associated with the user that are stored in database 140. In some cases, the user may have access and / or limited privileges to update, edit, delete, and / or add video streams associated with your user profile (for example, user-specific and / or similar contextual video streams). In some cases, the user may, for example, update and / or modify permissions and / or access associated with specific user video streams associated with that user. For example, in some cases, the user can redistribute, share, and / or save data associated with the user. In other cases, the user may block access to specific user and / or similar data. In some cases, the user may redistribute and / or share content, data, and / or video streams otherwise shared with the user (for example, which may or may not be associated with the user).
[0043] Returning to Figure 2, as described above, the processor 120 of the host device 110 can be configured to execute specific modules. The modules can be, for example, modules
Petition 870180048519, of 06/07/2018, p. 29/90
26/68 hardware, software modules stored in memory 115 and / or executed in processor 120, and / or any combination thereof. For example, as shown in Figure 2, processor 120 includes and / or performs an analysis module 121, a database module 122, a display module 123 and a location module 124. As shown in Figure 2, the analysis module 121, database module 122, presentation module 123, and the location module can be connected and / or electrically coupled. As such, signals can be sent between the analysis module 121, the database module 122, the presentation module 123, and the location module 124.
[0044] Analysis module 121 includes a set of instructions that can be executed by processor 120 (or its portion) that are associated with receiving / or collecting data associated with a user's facial recognition and / or a video stream. More specifically, the analysis module 121 can be operatively coupled to and / or otherwise in communication with the communication interface 125 and can receive data from it. Such data may, for example, be associated with a user (for example, facial recognition information, profile information, preferences, activity records, location information, contact information, calendar information, social media activity information, etc.), a location (e.g., location data, resource data, event scheduling), an event, and / or the like. As described in more detail here, the analysis module 121 can receive a signal from the communication interface 125 associated with a request and / or an instruction to operate and / or execute any number of processes associated with facial recognition.
[0045] In some cases, the analysis module 121 can receive data from the communication interface 125 in substantially time
Petition 870180048519, of 06/07/2018, p. 30/90
27/68 reais. This means, in some cases, an electronic device included in system 100 (for example, the client device 150) can be manipulated by a user to define and / or update data associated with the user's facial recognition and once defined and / or updated can send data to host device 110 over network 105. Thus, communication interface 125 can, when receiving data, send a signal to analysis module 121, which receives data in a period of time very short after being defined and / or updated by the electronic device. In other embodiments, the analysis module 121 can receive data from the communication interface 125 at a predetermined rate or similar based on, for example, an aggregation process, a current and / or predicted processor, memory, and / or load of network, and / or the like. [0046] As described above, the analysis module 121 can be configured to receive, aggregate, analyze, classify, parse, change, and / or update data associated with a facial recognition process or similar. More specifically, in some cases, a user can manipulate the client device 150 to capture one or more images or video streams of his face (as described in more detail here) and, in turn, can send signals associated with and / or representing the image data for the host device 110, for example, over the network 105. In some cases, the communication interface 125 can receive the image data and can send an associated signal to the analysis module 121. Upon receipt , analysis module 121 may execute a set of instructions or code (for example, stored in analysis module 121 and / or in memory 115) associated with aggregating, analyzing, classifying, updating, parsing, and / or otherwise process the image data. More specifically, the analysis module 121 can execute any process and / or algorithm of
Petition 870180048519, of 06/07/2018, p. 31/90
28/68 adequate facial recognition such as, for example, Principal Component Analysis using Eigenfaces (for example, Eigenvector associated with facial recognition), Linear Discrimination Analysis, Coincidence of Elastic Bunch Graph using Fisherface algorithm, Hidden Markov model, Learning from Multilinear subspace that uses tensor representation, coincidence of neuronal motivated dynamic connection, convolutional neural networks (CNN), and / or the like or their combination. In some implementations, the image data that the user provides for the host device 110 can be used in subsequent facial recognition processes to identify the user, through the analysis module 121.
[0047] The analysis module 121 can define a user profile or similar that includes the user's image data, and any other appropriate information or data associated with the user such as, for example, an image, video recording and / or audio recording, personal and / or identification information (for example, name, age, gender, birthday, hobbies, etc.), calendar information, contact information (for example, associated with the user and / or the user's friends , family, associates, etc.), device information (for example, a media access control (MAC) address, Internet Protocol (IP) address, etc.), location information (for example, current location and / or historical location data), social media information (for example, profile information, username, password, friends or contact lists, etc.), and / or any other appropriate information or data. As such, analysis module 121 may send a signal to database module 122 indicative of an instruction to store user profile data in database 140, as described in more detail herein.
[0048] In some cases, the analysis module 121 can receive
Petition 870180048519, of 06/07/2018, p. 32/90
29/68 video stream data (or image data, for example, from a photograph) and can be configured to analyze and / or process the video stream data to determine whether a portion of the video stream data matches any appropriate portion of user image data. That is to say, the analysis module 121 can use previously stored user image data as a template against which the data included in the video stream is compared. In other words, the analysis module 121 performs a facial recognition and / or analysis process on the video stream data based at least in part on the previously stored user image data. In some embodiments, the host device 110 and more specifically, the communication interface 125 receives the video stream data from the image capture system 160 either directly (for example, from one or more cameras over the network 105) or indirectly ( for example, from a computing device over network 105, which in turn is in communication with one or more cameras). In some embodiments, the analysis module 121 may be configured to analyze and / or process video stream data based at least in part on separating, parsing, classifying, and / or otherwise deconstructing the stream data video in its individual frames (for example, a still image at a predetermined time during the video stream). As such, the analysis module 121 can compare and / or analyze the data included in the video stream frame against the previously stored user image data.
[0049] In some cases, the analysis module 121 can also analyze the video stream data to determine the contextual information associated with the video stream such as, for example, location, location, time, coincident event (for example, a es team
Petition 870180048519, of 06/07/2018, p. 33/90
30/68 possession scoring a goal, being captured, for example, in a kiss cam, etc.), and / or any other appropriate contextual information. In some cases, the analysis module 121 may be configured to match, aggregate, and / or otherwise associate at least a portion of the video stream with the contextual data. For example, in some cases, video stream data may represent, for example, a user at a sports event. In such cases, contextual data can be, for example, a video stream from the sport or game event, and can include data associated with a time, location, location, teams, etc. As such, analysis module 121 may be configured to aggregate video stream data and contextual data so that the video stream data and contextual data substantially overlap (for example, occur and / or capture substantially associated data at the same time). In other cases, contextual data may include data associated with any other suitable context. In some cases, the analysis module 121 may be configured to use the contextual information associated with the video stream, along with data relating to a user's location, to additionally connect the video stream to a specific user. Analysis module 121 can be configured to compare contextual information to a user's location before comparing the data included in the video stream with the previously stored user image data (see Figures 5 and 6 for more details).
[0050] If the analysis module 121 determines that at least a portion of the data in the video stream meets a criterion (for example, the previously stored user image data matches a predetermined and / or acceptable probability), the analysis module analysis 121 can send one or more signals to the database module 122 indicative of an instruction to store
Petition 870180048519, of 06/07/2018, p. 34/90
31/68 at least the portion of the image and / or video stream data in database 140 and associate and / or otherwise store this data with the previously stored user image data. In some cases, the analysis module 121 can send signals to the database module 122 so that individual frames are stored in database 140, which in turn can be subsequently retrieved and processed to define a flow of data. video. In other cases, the analysis module 121 may send one or more signals to the database module 122 so that the portion of the video stream data is stored in database 140. That is, the analysis module 121 can at least partially redefine and / or reconstruct the video stream of the individual frames (which have been separated or deconstructed as described above).
[0051] In some cases, the host device 110 can receive video stream data (for example, from the image capture system 160 and through the network 105 and the communication interface 125) and the analysis module 121 and / or any other suitable module not shown in Figure 2, can perform one or more pre-processing and / or pre-classification procedures before performing the facial recognition process (just described). For example, in some embodiments, analysis module 121 (or another module) can analyze video stream data to determine how and / or define a data set that includes, for example, identifying information and / or contextual information such as location, time, event, etc. Once defined, analysis module 121 can analyze user data stored in database 140 (for example, by sending a signal to database module 122 indicative of an instruction to query database 140 and / or similar) to determine whether a portion of data associates Petition 870180048519, of 06/07/2018, p. 35/90
32/68 to a user meets a criterion (s) such as matching the data set including the contextual information associated with the video stream.
[0052] In some cases, the criterion (s) may be associated with a level of confidence and / or coincidence limit, represented in any suitable way (for example, a value such as a decimal, a percentage, and / or similar). For example, in some cases, the criterion (s) may be a limit value or similar such as a 70% match of the video stream data and at least a portion of the data stored in the database, a match 75% of the video stream data and at least a portion of the data stored in the database, a match of 80% of the video stream data and at least a portion of the data stored in the database, a match of 85 % of video stream data and at least a portion of the data stored in the database, a 90% match of the video stream data and at least a portion of the data stored in the database, a 95% match video stream data and at least a portion of the data stored in the database, a 97.5% match of the video stream data and at least a portion of the data stored in the database, a 99% match video stream data and at least a portion of the data stored in the database, or any percentage between them.
[0053] In some cases, the data associated with the user may include, for example, calendar data, location data, preference data, and / or the like. If, for example, the data does not meet the criterion, the analysis module 121 can define an indication that the data associated with that user can be excluded, for example, from the facial recognition process. In this mode, the prePetition 870180048519, of 06/07/2018, p. 36/90
33/68 processing and / or pre-sorting can reduce an amount of processing load or the like during the facial recognition process. Although described above as querying database 140 for user data, in some embodiments, host device 110 can send a signal to a device associated with the user (eg client device 150) indicative of a request for location or similar data associated with that device. When receiving location data (for example, global positioning service (GPS) data from the device, using location information and / or features, such as a landmark and / or background scenery, within an image or video, etc.) or similar, the analysis module 121 can determine whether the location data matches the location data associated with the video stream, as described above.
[0054] For example, in some cases, the analysis module 121 can receive video stream data from a sport event that also includes location data associated with, for example, an arena. In response, the analysis module 121 can send a request for location data from a client device (e.g., client device 150) associated with a user. If, for example, the location data associated with the video stream and the location data associated with the client device are substantially similar (for example, the location data associated with the video stream and the location data associated with the client device indicate that the source of the video stream and the client device are and / or were within a predetermined distance from each other) and / or the location data associated with the client device is within a predetermined range of data values from location or the like, analysis module 121 can increase a confidence score and / or otherwise consider the
Petition 870180048519, of 06/07/2018, p. 37/90
34/68 result as contributing to finding the limit and / or otherwise satisfying the criterion (s). Location data can be, for example, geolocation data based on GPS, location and / or network data (for example, via NFC verification, Bluetooth verification, cell triangulation, switching and / or communication protocols) cognitive network, etc.), social network data such as a checkin, and / or the like. For example, location module 124 can process location data in order to identify the location of the video stream and / or the user, and provide data for analysis module 121 to allow analysis module 121 to modify the confidence score. In this mode, the confidence score can be calculated based on the location data.
[0055] In other implementations, the location module 124 can process the location data and can provide the location data processed for the analysis module 121 when the location data associated with the video stream and the location data associated with the user are substantially similar (for example, the location data associated with the video stream and the location data associated with the client device indicate that the source of the video stream and the client device are and / or have been within a predetermined distance a from the other). The analysis module 121 can then generate and / or modify a confidence score based on the location data and a facial recognition analysis of the video stream. In this mode, the confidence score can be generated and / or modified when the location data associated with the video stream and the location data associated with the user are determined to be substantially similar and cannot be generated and / or modified when the data of location associated with the video stream and the location data associated with the user are not substantially similar. Yet, in this
Petition 870180048519, of 06/07/2018, p. 38/90
35/68 mode, the confidence score can be calculated as a result of both an analysis of location data and a facial recognition analysis. More details on location module 124 can be found at least in Figures 5-6. In this mode, the host device 110 (for example, through the analysis module 121) can determine, for example, a proximity of a client device to a location where the video stream data was captured.
[0056] Although described as analyzing location data, in other cases, the analysis module 121 can analyze data associated with any source, activity, location, pattern, acquisition, etc. appropriate. For example, in some cases, analysis module 121 can analyze ticket sales associated with a location. In other cases, the analysis module 121 can analyze social media posts, comments, likes, etc. In some cases, the analysis module 121 can collect and / or analyze data associated with a user (as described above) and can define, for example, a user profile that can include, inter alia, user identification data, data facial recognition data, customer device data, purchase data, internet web browsing data, location data, social media data, preference data, etc. Thus, user profile data can be analyzed to determine a confidence score, value, and / or indicator, which can be assessed against a threshold score, value, and / or indicator to determine whether user data and / or the video stream data meets the criteria (s). Consequently, in such modalities, non-facial recognition data (for example, ticket sales data, social media posts, and / or features such as an individual's clothing in a video or image, location data such as landmarks within image, background of given backgrounds, etc.) can be used
Petition 870180048519, of 06/07/2018, p. 39/90
36/68 to corroborate the facial recognition data and / or increase I decrease a confidence score.
[0057] Although the analysis module 121 is described above as analyzing the video stream data to define the facial recognition data and contextual data associated with the video stream, in other modalities, the facial recognition process and the data process contextual can be performed separately and / or independently. For example, in some modalities, the analysis module 121 may be configured to perform the facial recognition process as a different module, processor, device, server, etc. it can be configured to run the contextual data process. For example, the location module 124 can perform an image and / or video stream analysis based on the location data, image characteristics, and / or the like. Thus, time to analyze the video stream data can be reduced and / or the processing load can be distributed when compared to the facial recognition process and the contextual data process being performed by the same module.
[0058] As described above, database module 122 includes a set of instructions executed by processor 120 (or its portion) that is associated with monitoring database 140 and / or updating data stored therein. For example, database module 122 may include instructions to have processor 120 update data stored in database 140 with at least a portion of the facial recognition data received from analysis module 121. More specifically, the database module 122 can receive, for example, the user image data associated with the user of analysis module 121 and, in response, can store user image data in database 140. In some cases, the da bank module
Petition 870180048519, of 06/07/2018, p. 40/90
37/68 of 122 can receive a signal from analysis module 121 indicative of a request to query database 140 to determine whether the data stored in database 140 and associated with user image data for the user matches any adequate portion of the video stream data, as described above. If, for example, at least a portion of the video stream data meets a criterion (s) (referred to hereinafter as a criterion for simplicity and not for the exclusion of multiple criteria), the database module 122 may be configured to update data stored in database 140 associated with that user. That is, if at least a portion of the video stream data matches the user image data within a predetermined or similar probability. If, however, the video stream data does not match the user image data stored in database 140, database module 122 can, for example, query database 140 for the next entry (for example, example, data associated with the next user) and / or may otherwise not update database 140. Furthermore, database module 122 may be configured to store data in database 140 in a relational based mode (for example, database 140 may be a relational database and / or the like) and / or in any other suitable manner.
[0059] Presentation module 123 includes a set of instructions executed by the processor (or a portion thereof) that is associated with the definition of a contextual video stream and / or a presentation that represents at least a portion of the video stream data satisfying the criterion during the facial recognition process, as described above. More specifically, presentation module 123 can be configured to define a contextual video stream and / or a presentation that represents an identifiable user
Petition 870180048519, of 06/07/2018, p. 41/90
38/68 (for example, through facial recognition) at an event, location, location, and / or the like. Once the contextual video stream is defined, the display module 123 can send a signal associated with the contextual video stream to the communication interface 125, which in turn can send a signal (for example, over the network 105) for client device 150 which is indicative of an instruction to graphically render the contextual video stream on its display.
[0060] Although the display module 123 and / or other portion of the host device 110 is described above as sending a signal to the client device 150 indicative of instruction to present the contextual video stream on the display of the client device 150, in in other cases, presentation module 123 can define the contextual video stream and can send a signal to database module 122 indicative of an instruction to store the contextual video stream in database 140. In such cases, data associated with the contextual video stream may be stored and / or otherwise associated with user data stored in database 140. In some cases, host device 110 may retrieve contextual video stream from database 140 in response upon request from client device 150 (and / or any other suitable device). More specifically, in some embodiments, the user can manipulate the client device 150 to access a web page on the Internet. After being authenticated (for example, by entering credentials or the like) the user can interact with the webpage so that a request for access to the contextual video stream is sent from the client device 150 to the host device 110. Thus, the host device 110 (for example, database module 122) can retrieve the contextual video stream from database 140 and can send a signal to the
Petition 870180048519, of 06/07/2018, p. 42/90
39/68 client device 150 operable in presenting the contextual video stream on the display (for example, rendering the contextual video stream via the Internet and the webpage). In other words, the contextual video stream can be stored in the cloud and accessed via a web browser and the Internet.
[0061] Although the analysis module 121, the database module 122, and the presentation module 123 are described above as being stored and / or executed in the host device 110, in other modalities, any of the modules can be stored and / or executed, for example, on the client device 150 and / or the image capture system 160. For example, in some embodiments, the client device 150 may include, define, and / or store a presentation module ( for example, as a native application). The display module can be substantially similar to or the same as the display module 123 of the host device 110. In such embodiments, the display module of the client device 150 can replace the function of the display module 123 otherwise included and / or executed on the host device 110. Thus, the presentation module of the client device 150 can receive, for example, a data set associated with a contextual video stream and when received, it can define a presentation to be presented in the display of client device 150.
[0062] Figure 3 is a flow chart illustrating a method 300 for defining a contextual video stream according to a modality. Method 300 includes receiving, on a host device and a client device over a network, a signal indicating a request to register facial image data associated with the user, at 302. For example, in some embodiments, the network may any suitable network or combination of networks such as, for example,
Petition 870180048519, of 06/07/2018, p. 43/90
40/68 example, the network 105 described above with reference to Figure 1. The host device can be substantially similar to or even that the host device 110 described above with reference to Figures 1 and 2. Similarly, the client device can be substantially similar to or the same as the client device 150 described above with reference to Figures 1-2. In some cases, the client device may be configured to capture initial facial image data and may send initial facial image data to the host device. Specifically, in some embodiments, the client device may be configured to capture an image or facial images of the user in any suitable mode. Consequently, the host device can receive facial image data from the client device and can perform any suitable or similar processes associated with a user's registration and / or the user's facial image data.
[0063] Method 300 includes recording the facial recognition data associated with the user and storing the facial recognition data in a database in communication with the host device, at 304. The database can be any suitable database such such as, for example, database 140 described above with reference to Figure 1. The recording of facial recognition data can include any suitable process, method, and / or algorithm associated with facial recognition such as those described above. In some cases, the host device may be configured to define user image or similar data based on facial recognition and may store at least a portion of the user image data in the database.
[0064] The host device receives contextual video stream data associated with an event and / or location, at 306. The host device can receive the context video stream data
Petition 870180048519, of 06/07/2018, p. 44/90
41/68 of an image capture system such as the image capture system 160 (for example, a camera and / or client device) described above with reference to Figure 1. More specifically, the host device can receive the contextual video stream data either directly (for example, from one or more cameras over the network) or indirectly (for example, from a computing device over the network, which in turn is in communication with one or more cameras).
[0065] In one example, the camera can record contextual video stream data, and can send contextual video stream data to the host device. In another example, a user can record a video through an application that runs on a client device being operated by the user (for example, through a User Generated Content (UGC) interface within the application that runs on the client device ). By starting the recording through the application (for example, clicking a Record button and / or similar in the UGC interface), the user can record a contextual video stream, such as which the client device can associate location data (for example , geolocation, Near Field Communication (NFC) data, Bluetooth communication data with other devices, cell triangulation, event and / or location check-in data, and / or network Wi-Fi connection information) with the contextual video stream. Specifically, the contextual video stream can be identified with the location data, and / or it can be associated with a data structure that encapsulates the location data.
[0066] The contextual video stream data is analyzed to determine whether the contextual video stream data meets a criterion associated with facial recognition of facial image data in the contextual video stream data, at 308. For example, the
Petition 870180048519, of 06/07/2018, p. 45/90
42/68 host device can receive contextual video stream data (or image data, for example, from a photograph) and can analyze and / or process contextual video stream data to determine whether a portion of the stream data contextual video matches any appropriate portion of the facial image data. That is, the host device can use the facial image data as a template against which the data included in the contextual video stream is compared. In other words, the host device performs a facial recognition and / or analysis process on the contextual video stream data based at least in part on the facial image data. In some cases, the criterion may, for example, be associated with a coincidence of the contextual video stream data with the facial image data with a predetermined and / or acceptable probability. In some embodiments, the host device may be configured to analyze and / or process contextual video stream data based at least in part on separating, parsing, classifying, and / or otherwise decoupling video stream data contextual in your individual frames (for example, a still image at a predetermined time during the video stream). As such, the host device can compare and / or analyze data included in the contextual video stream frame against facial image data.
[0067] In some cases, the analysis of the contextual video stream data also includes analyzing the contextual video stream data to determine the contextual information associated with the video stream such as, for example, location, location, time, coincident event (for example, a sports team scoring a goal, being captured, for example, on a kiss cam, etc.), milestones within the image, and / or any other appropriate contextual information.
Petition 870180048519, of 06/07/2018, p. 46/90
43/68
In some cases, the host device may be configured to match, aggregate, and / or otherwise associate at least a portion of the video stream with contextual data. For example, in some cases, video stream data may represent, for example, a user at a sports event. In such cases, contextual data can be, for example, a video stream from the sport or game event, and can include data associated with a time, location, location, teams, etc. As such, the host device may be configured to aggregate video stream data and contextual data so that the video stream data and contextual data substantially coincide (e.g., occur and / or capture data associated with substantially the same time). In other cases, contextual data may include data associated with any other suitable context.
[0068] A user contextual video stream is defined when the criteria associated with facial recognition of the facial image data in the contextual video stream data is satisfied, in 310. For example, when the host device determines that at least a portion of the data in the contextual video stream meets a criterion (for example, the facial image data matches a predetermined and / or acceptable probability), the host device can define the user's contextual video stream and can store the contextual video stream of the user in the database. With the user's contextual video stream defined, the host device sends a signal indicative of an instruction to present the user's contextual video stream on a display on the client device in 312 (for example, graphically rendering the contextual video stream on an interface instantiated on the client device). For example, in some embodiments, the host device can send a signal to the client device over the network, which is operable in
Petition 870180048519, of 06/07/2018, p. 47/90 display the user's contextual video stream on the display of the client device. In other embodiments, the host device may store the contextual video stream (for example, in the database or the like) and may be configured to retrieve the contextual video stream from the database user in response to a request from the customer (and / or any other suitable device). More specifically, in some modalities, the user can manipulate the client device to access a webpage on the Internet. After being authenticated (for example, by entering credentials or similar) the user can interact with the webpage so that a request for access to the contextual video stream is sent from the client device to the host device. Thus, the host device can retrieve the contextual video stream from the database and can send a signal to the operable client device to present the contextual video stream on the display (for example, graphically rendering the contextual video stream over the Internet and the webpage). In other words, the contextual video stream can be stored in the cloud and accessed via a web browser and the Internet.
[0069] In other implementations, when a contextual video stream meets the criteria (for example, when the contextual video stream matches the user's facial image data at a predetermined probability, and / or the like), the host device can automatically send the contextual video stream data to the user. For example, in some implementations, the user may also be operating a client device by instantiating an application that is tracking user location data for this user. When an image capture device (for example, such as a standalone camera and / or another user) records the contextual video stream data, the host device can
Petition 870180048519, of 06/07/2018, p. 48/90
45/68 to conclude that the contextual video stream data coincides with the user based on a facial analysis of the contextual video stream and facial image data associated with the user. The user's client device can also send location data associated with the user and the client device to the host device. The host device can refine, using both facial analysis and location information, the probability that the user appears in the contextual video stream. If the probability that the user appears in the contextual video stream meets a criterion (for example, exceeds a predetermined threshold, and / or the like), the host device can send the contextual video stream data to the user. Alternatively, the host device can pre-filter the contextual video stream based on the location information, so the probability is calculated when the user's location information is substantially similar to the location information of the contextual video stream, and not calculates the probability when the location data of the contextual video stream is not substantially similar to the user's location information.
[0070] In other implementations, when a contextual video stream meets the criteria (for example, when the contextual video stream matches the user's facial image data at a predetermined probability, and / or the like), the host device can store the contextual video stream data and associate the contextual video stream data with the user based on the user’s interaction with the video. For example, in some implementations, the user can access an application instantiated on a client device associated with the user, to search and / or access contextual video stream data. The user for, for example, viewing the contextual video stream data within the user profile of another user associated with that user, and / or can search for da
Petition 870180048519, of 06/07/2018, p. 49/90
46/68 of contextual video streams to view within an application interface. When the user accesses the contextual video stream data within the application, the application can send a signal to the host device indicating which user is accessing that contextual video stream data. The host device can automatically determine whether or not a facial analysis of the contextual video stream data was performed based on the facial image data associated with that user, and can automatically perform a facial analysis of the contextual video stream data, based on the that user's facial image data, if the user's facial image data has not been previously compared with the contextual video stream data. In this mode, the host device can delay processing the contextual video stream data to identify users within the contextual video stream data, until users attempt to access the contextual video stream data.
[0071] In some cases, a user can search for an event and check in at that event after the event. For example, the user can identify an event (for example, viewing a list of events, viewing the location of events on a map, etc.) and can select an event. Based on the event's user selection, the host device can perform a facial analysis of the video and / or image streams associated with that event based on that user's facial image data. If the host device identifies a video and / or image stream that includes the user (for example, with a predetermined probability), the host device can provide such video and / or image streams to the user.
[0072] Although method 300 is described above as sending and / or receiving video streams, image data, contextual data, etc. and presenting and / or sharing video and / or data streams
Petition 870180048519, of 06/07/2018, p. 50/90
47/68 user-specific image with one or more users, it should be understood that a system can be arranged so that video stream data and / or image data can be captured in any suitable mode, analyzed by any device appropriate, and sent to and / or shared with any appropriate user or user device. By way of example, in some cases, a user can manipulate a user device (for example, a client device such as a client device 150) to capture a facial image of the user. For example, the user can open a mobile application (for example, when the user or client device is a smartphone or other mobile or usable electronic device) and can capture a facial image (for example, a selfie) through a camera. client device. In other words, the user can control the camera of the client device through the application to capture a selfie. Such a selfie can be provided to register a user so that the application can identify the user's facial recognition data (for example, facial appearance characteristics). This facial recognition data can be used to identify the user in videos and / or images subsequently received.
[0073] In some cases, the user can capture content (for example, image data and / or video stream) through the application. As described above, the content can be a video stream of one or more people in a given context such as, for example, one or more people at a sports event or similar. In some cases, the user's captured (for example, generated) content can be associated with contextual data such as time, date, location, location, event, etc. and / or can otherwise be identified with data and / or metadata. In other cases, user-generated content does not have to be associated with contextual data. The content
Petition 870180048519, of 06/07/2018, p. 51/90
48/68 User generated data (for example, video stream data or the like) can be analyzed through facial recognition and / or other image analysis via the client device or a host device to determine the presence of any registered user (for example, any user with a user profile stored in the database). If a registered user is identified in the video stream, the user, the client device, and / or the host device can define a user-specific video stream associated with one or more of the identified users. The user, the client device, and / or the host device can then determine they can then determine whether to share the user-specific video stream with each identified user. In some cases, the sharing of the user-specific video stream (s) may be automatic based on a user profile and / or preference and / or based on an adjustment or similar within the mobile application or account. In other cases, sharing the user-specific video stream (s) may be based on manual input or other user input (for example, based on a section or similar). In still other cases, sharing of the user-specific video stream (s) may be based on a peer network section, in which each user (or each client device used in the peer network section) receives a user-specific video stream. In this mode, the user generated content (for example, the video stream and / or image data captured by the user) can be captured, analyzed, and / or shared in a similar way to those described here.
[0074] Figure 4 is a flowchart that illustrates a method for presenting a contextual video stream for, for example, a mobile device associated with a user according to a modality. In some cases, video file (s) and / or photo file (s) may
Petition 870180048519, of 06/07/2018, p. 52/90
49/68 can be loaded into a 485 media loader. The 485 media loader can be any suitable device configured to receive and / or process video and / or image files such as, for example, the host device 110 described above with reference to Figures 1 A, 1B and 2. A master video and / or photo file is then stored in a 486 master media storage. The 486 master media storage can be any suitable storage device. For example, the master media storage 486 can be included in and / or a part of memory included in the media loader 485. In other embodiments, the master media storage 486 can be a database or the like such as, for example , the database 140 described above with reference to Figures 1A and 1B.
[0075] In some cases, the master video file can be sent from the master media storage 486 to a 487 video encoder. The 487 video encoder can be any suitable device or portion of a device configured to convert the video file master in one or more desired formats. For example, as shown in Figure 4, the 487 video encoder can convert the master video file into a facial recognition video and a mobile compatible video file, each of which is stored in the 486 master media store. A list of one or more facial recognition video files and / or photo files is then sent to a 488 workflow driver, who can prioritize, organize and / or otherwise control an order in which the files are subsequently processed and can send an operable signal to start processing the facial recognition video file (s) and / or photo file (s) to a 491 face matching and detection processor (for example, a processor, module , device, etc. such as, for example, the module
Petition 870180048519, of 06/07/2018, p. 53/90
50/68 of analysis 121 described above with reference to Figure 2), as described in more detail herein. In addition, an indication associated with the workflow can be sent from the workflow driver 488 to a database 493, which can store the indication associated with the workflow and which can send data associated with the indication to a processor. 494 web service (for example, an Internet website service provider, processor, module, and / or device), as described in more detail here.
[0076] As shown in Figure 4, the mobile compatible video file is sent from the master media storage 486 to a video clip cutter 489, which can also receive data associated with recognition events, as described in more detail here. The master video file or photo file is sent from the master media storage 486 to a 490 miniature resizer, which can also receive data associated with recognition events, as described in more detail here. The facial recognition or photo video file (s) is / are sent from the master media storage 486 to the 491 face matching and detection processor, which in turn can perform any suitable facial recognition process to define recognition events. Furthermore, the 491 face detection and matching processor can analyze and / or process the facial recognition video and / or photo file according to the priority and / or order defined by the workflow driver 488.
[0077] As described above, the data associated with the recognition events can then be sent from the face detection and matching processor 491 to the video clip cutter 489 and the miniature resizer 490. The video clip cutter 489 can be any processor, module, and / or suitable device that can receive the mobile compatible video file and that can subscribe 870180048519, of 06/07/2018, p. 54/90
51/68 to trim, cut, extract, separate, and / or otherwise define a video clip associated with a user's recognition events within the facial and / or photo recognition video. The video clip associated with the user recognition event can then be sent from the video clip cutter 489 to a compatible mobile media storage 492. The thumbnail resizer 490 can be any suitable processor, module, and / or device that can receive the master video and / or photo file (s) and which can subsequently define one or more thumbnails (for example, small images with a relatively small file size, which in turn can be associated with and / or indicative of a larger image and / or video). In this mode, the thumbnails can be associated with and / or indicative of recognition events and can be sent from the 490 miniature resizer to the 492 mobile compatible media storage.
[0078] As shown in Figure 4, video clips and thumbnails can be sent from 492 compatible mobile media storage, for example, to one or more mobile applications and / or 495 websites. For example, in some cases, video clips and thumbnails can be uploaded to an Internet server or similar, which in turn can display video clips and thumbnails on a website or similar. In other cases, video clips and thumbnails can be sent to a client device associated with the user, which in turn, can display video clips and thumbnails on a display (for example, when a mobile application is opened, selected, running, etc.). Furthermore, metadata (for example, user identity, event identity, event location, location of a client device, etc.) or the like associated with the workflow indication (described above) can be sent from the service processor web 494 for mobile application and / or websites
Petition 870180048519, of 06/07/2018, p. 55/90
52/68
495. In this mode, a video clip of a user and any contextual and / or metadata associated with it can be sent to and / or accessed by the user through a mobile application and / or website.
[0079] Figure 5 is an illustration of an image capture system 560 (for example, similar to the image capture system 160 shown in Figure 1) that captures contextual information on media, according to a modality. Initially, the 560 image capture system can capture images and / or video from a location. The 560 image capture system can identify features such as background landmarks, unique features of walls, floors, design elements, furniture, and / or the like within the images and / or video of the location. The image capture system 560 can send these characteristics (also referred to herein as milestone data and / or information) to the host device 510 and the host device 510 can store this information (for example, within a database). The host device 510 can store this information associated with location information of the location. Similarly stated, host device 510 can store landmark information so that it is associated with the location of that landmark within the site.
[0080] In some implementations (as described in Figures 1-4), the 560 image capture system can capture a medium (for example, a video stream, photographs, and / or other media) that includes a 502 user. User 502 can use a mobile device 504 that includes a mobile application configured to send location data 506 (for example, Global Positioning System (GPS) coordinates, Wi-Fi signal that indicates it is within range of a access, NFC signal information, Bluetooth communications that indicate it is within range of an IBeacon, cell triangulation information, switching information and
Petition 870180048519, of 06/07/2018, p. 56/90
53/68 cognitive network protocol that estimates a distance from a content capture point, location data associated with a position at a location such as a seat or section number, and / or similar location data) to the host device 110, for example, when the mobile device detects a Wi-Fi signal and / or network associated with a location. In some implementations, the mobile application may be configured to interact with an iBeacon (and / or a similar device configured to transmit information to devices), and may be configured to send location data to the host device 110 (for example, such as the iBeacon identifier, mobile device GPS data, and / or other such information).
[0081] The 508 media (for example, photographs, videos, and / or relative media files) captured by the image capture system 160 may include an image or video of the user 502, as well as buildings, site features, objects, background landmarks, and / or other aspects of the background 510 of the scene. For example, the media may include not only user 502, but seats next to user 502 at a sports venue, landmarks at the back of the media and associated with a specific location (for example, within a venue), signs, and / or other such information (for example, unique characteristics of walls, floors, design elements, furniture, etc.). The host device 110 can use the background with the user's location data to further verify that the user 502 is likely to appear on the media. More specifically, in some cases, an analysis module of the host device 510 (for example, similar to the analysis module 121 shown in Figure 2) can perform image processing on the video stream, for example, to extract a scenario and / or other fund, non-person data from the video stream (also referred to as milestone data). For example, the po analysis module
Petition 870180048519, of 06/07/2018, p. 57/90
54/68 to use image processing techniques to detect a seat number 200 on the media. A location module of the host device 510 (for example, similar to 124 shown in Figure 2) can match the extracted data with the location data in the database (for example, using metadata, passwords, and / or image processing from previously stored location images) to estimate a location of the video stream. For example, the location module can use a seat number 200 to estimate the seat 514 on which the user 502 appears to sit on the media, and can determine an approximate location in which the media was captured based on the location of the seat within the location . For another example, the location module can compare the landmark data in the video stream with the images and / or video previously captured and stored by the host device 510. Due to the association between the location and the landmark data stored by the host device 510 , the host device 510 can identify a location of the video stream.
[0082] In some cases, a user can also check in on the spot. Specifically, the user's mobile device can send a message that includes location information to the user (for example, the GPS coordinates of the user's mobile device, an identifier for an iBeacon, NFC identification, cellular network, Wi-Fi and / or other network, and / or similar device in close proximity to the mobile device, and / or the like). The location module can store location data in the user's account data. After the location module has determined a location of the video stream and the user, the location module can compare the seat 208 with the location data provided by the user's mobile device to determine the likelihood that the user 502 was actually sitting on the seat 208. For example, the
Petition 870180048519, of 06/07/2018, p. 58/90
55/68 location module can retrieve recordings from users whose most recent location data closely matches the estimated media location (for example, whose most recent location data is within a predetermined number of meters from the estimated media location, and / or similar). For each retrieved user recording, the analysis module can perform facial recognition on the media, for example, using the user's image data for comparison, to determine whether the user appears on the media. Based on this information, the host device can determine a list of users whose image could have been captured on the media at a specific location, and can determine, from this small group of users, whether a positive match between people in the media, and any of the 502 users in the user list, was made. In some cases, a facial recognition can then be performed for the image to identify which users in the small user group (for example, users identified as being in the general area based on milestone information and / or device location information) user) are identified in the media. Reducing the user group using milestone data and user device location information reduces the number of false positives when using facial recognition. In some implementations, the host device may use facial recognition analysis, and location data, to determine whether or not to store and / or discard (for example, delete and / or not store) the video stream in the database. In other implementations, the host device can store the video stream, regardless of whether or not the video stream can be matched to a specific user. The host device can then associate the video stream with a specific user when the user's device location, and when recognition data
Petition 870180048519, of 06/07/2018, p. 59/90
56/68 facial features associated with the user, are used in combination with the frame and / or other location data, to determine whether or not the user can be identified in the video stream.
[0083] Figure 6 is a logical flowchart that illustrates using contextual information on media, and location data, to identify a user on the media, according to a modality. In some implementations, for example, user 504's mobile device can check in at one location and / or another location at 602 (for example, by sending location data and / or iBeacon identifiers to host device 110 (shown in Figure 1)). This can provide the same location and / or a host device associated with the video recognition system (for example, the host device 110 in Figure 1) an indication that the user is within the location and / or at the event. In addition, this can provide an indication of a user's location within the venue and / or at the event. In some implementations, the mobile device 504, via the mobile application stored on the device, may be configured to periodically send updated GPS data to the host device 110, and / or may be encouraged to send location data to the host device 110 when the 504 mobile device is within close proximity to an iBeacon, Wi-Fi hotspot and / or similar device. Host device 110 can store 604 location data in database 140.
[0084] In some implementations, instead of the user's mobile device checking in, a ticket sales and / or ticket processing device at a location can send a message to the host device 110 indicating that a user has purchased and / or used a ticket for a specific event on the spot, the time at which the ticket was purchased and / or redeemed, and / or the like. In other implementations, a user's location can be inferred, for example
Petition 870180048519, of 06/07/2018, p. 60/90
57/68 example, based on location data previously stored for the user, based on tickets the user has purchased for events at specific locations, and / or similar.
[0085] An image capture system 160 (shown in Figure 1) can capture media (for example, including but not limited to recording a video extension and / or capturing at least one photograph), at 606, and can send the media to the host device 110 (shown in Figure 1), at 608. The image capture system 160 can also send its location data (for example, GPS coordinates to the image capture system 160, and / or similar) to host device 110. Host device 110 can identify a location on the media, at 610 (for example, using landmark data in the media background and / or background). For example, in some implementations, host device 110 (for example, through analysis module 121 shown in Figure 1) may use image recognition processing techniques to detect specific objects in the background (seats, local landmarks, and / or similar ), to detect background identification information (for example, plates, seat numbers, location characteristics, and / or the like), and / or estimate a distance between the image capture device 160 and the user 502 (for example , using the size of objects in the media, in relation to the location of the image capture system and / or user 502, to estimate the distance). The host device 110 (for example, via the location module 124) can then use the identification information, objects, and / or distance to identify the location captured on the media. For example, if the analysis module 121 detects a seat number at a sports location, the location module 124 can use the location data from the image capture system to determine at which sports location the image capture system 160 is located 870180048519, from 06/07/2018, p. 61/90
58/68 and can retrieve a location map and / or other data to determine where on the site a seat with the seat number would be located. As another example, location module 124 can detect a national landmark (for example, a famous statue) and / or a status plate, and can determine a GPS location for the user based on known location data for the landmark national and / or state board. If the image capture device 160 provides location data, the location module 124 can verify the detected location based on the location data from the image capture device.
[0086] In some implementations, location module 124 can also determine a location based on other media previously stored on host device 110. For example, image capture system 160 can record a video that includes a famous statue, and the location module 124 can determine the GPS coordinates for the statue and store said coordinates, for example, as metadata for video data as stored in database 140. If the image capture device 160 subsequently sends media that also includes the statue, the location module 124 can detect, using image processing techniques similar to those described here, the identity of the statue using the previously received video data, and can determine the statue's location using the video data previous (for example, through metadata stored with video data, and / or similar s). For another example, the location module can use pre-captured image data of landmarks within a location that associates landmarks with a location within the location to identify the location within the location captured on the media.
[0087] The location module 124 can then retrieve those from the
Petition 870180048519, of 06/07/2018, p. 62/90
59/68 user location data (eg GPS data, iBeacon data, ticket acquisition data, and / or the like) for users in database 140 out of 612. For each 614 user, the host device can map the user's location data to a location on site and / or the event at 616. For example, if the user location data indicates that the user is at a specific sport location, location module 124 can map the user location data for a location within the location, for example, using a location map and / or similar data. Location module 124 can then determine 618 whether or not the user's location on the site matches the location that location module 124 has identified on the media. For example, the host device 110 can determine whether a seat number detected on the media matches a seat number near the iBeacon identified in a check-in message from the user's mobile device, and / or whether the seat number is in immediate proximity to a seat number associated with the user's ticket. If the two locations do not match, the location module 124 determines that the user is probably not in the location where the media was recorded, and the location module 124 can analyze 620 the location data of the next user.
[0088] If the two locations coincide, analysis module 121 (for example, shown in Figure 2) can perform facial recognition 622 on the media, for example, using the user's previously stored image data and the media received from image capture system 160. If the analysis module 121 detects a match 624 between the user and a person on the media, the host device can store 626 the media (for example, including metadata such as the location where the media was recorded , an identifier associated with the user, and / or other information). O
Petition 870180048519, of 06/07/2018, p. 63/90
60/68 host device 110 can then notify the user (eg via email, text message (eg Short Message Service (SMS) and / or Multimedia Message Service (MMS)), notification mobile device application, and / or the like) that the image capture system 160 captured a media that includes the user. The user can then access the media. If the two locations do not match, the analysis module may not perform facial analysis, and may end the process of matching the user to the media.
[0089] In some implementations, the location module 124 can perform the location analysis before performing facial recognition on the media. In other implementations, host device 110 can perform location analysis after performing facial recognition on the media. Performing location analysis before facial recognition can reduce the number of comparisons made (thereby reducing the amount of time and resources used to perform facial recognition), and can reduce the amount of data retrieved and processed from database 140. This it can also reduce the number of false positives produced by the facial recognition process since the facial recognition analysis can be performed on those individuals whose location matches the location of the image and not on individuals whose location does not match the location of the image.
[0090] In some cases, a facial recognition confidence score can be calculated based on the location information identified by the milestone data in the media. For example, if a milestone in the media indicates that the video is a specific portion of a location and a user's device indicates that the user is within that portion of the location, the confidence score that the user is within the media may increase. On the contrary, if a milestone in
Petition 870180048519, of 06/07/2018, p. 64/90
61/68 media indicates that the video is from a specific portion of a location and a user's device indicates that the user is not within that portion of the location, the confidence score that the user is within the media may decrease. Thus, while not limiting the number of individuals on which facial recognition is performed, the milestone data can reduce false positives by affecting users' confidence scores.
[0091] Although described above as being used in conjunction with facial recognition, in other modalities, the location information received from the user's device and the location information derived from the landmark data in the image and / or video can be used without facial recognition to identify a user in the video. Specifically, for example, the location module (for example, shown in Figure 2) can determine to use the information in the video (for example, use information in the background and / or media background) of a location in the video. If a user device indicates that a user is in that specific location, the user can be identified as being included in the video. The video can then be provided to the user, as described above.
[0092] Although described above as receiving location information from an image capture device (for example, a position with the location), in other embodiments such location information is not received and the location of the image capture device may be identified only on the basis of milestone data in the media (for example, using information in the media background and / or background). In such embodiments, the image capture device not associated with the video recognition system (for example, the video recognition system 100 in Figure 1) and / or image capture devices not communicatively coupled with the video recognition system video can be used Petition 870180048519, from 06/07/2018, p. 65/90
62/68 to capture images and / or videos. The location of such images can be identified without location specific data (other than the image itself) being provided for the image capture device.
[0093] Although several modalities have been described above, it must be understood that these were presented by way of example only, and not limitation. For example, although the modalities and methods have been described here as defining a user's contextual video stream at an event or similar and sending the contextual video stream to a client device and / or otherwise allowing access to the stream. contextual video through, for example, a web browser and the Internet, in other modalities, a host device can store, in a database, any number of contextual video streams associated with a user. In some cases, the host device may be configured to define a user profile or the like that can include any number of user contextual video streams. In some cases, the user can access his user profile through a mobile application, a computer application, a web browser and the Internet, and / or the like. Furthermore, in some cases, the user may share or otherwise request the host device to share any number of the user's contextual video streams with a different user and / or through a social media site. In some cases, a user may allow access to a portion of his user profile so that other users can view the contextual video streams included in this.
[0094] Although specific examples have been specifically described above, the modalities and methods described herein can be used in any suitable way. For example, although system 100 is described above as defining a video stream conPetição 870180048519, from 06/07/2018, p. 66/90
63/68 textual information of a user at a sport event, in other modalities, the methods described here can be used to identify an individual using, for example, facial recognition and video analytics in any suitable environment, location, arena, event, etc. For example, in some modalities, the methods described above can be used to capture a contextual video stream at a concert, a rally, a graduation, a party, a shopping mall, a business location, etc. In one example, a host device can receive a contextual video stream from, for example, a graduation. In some cases, as described above, the host device can perform any facial recognition and / or video analytics suitable to identify the trainee (and / or any individual and / or user). Furthermore, the host device can be configured to analyze contextual information such as, a user profile associated with the trainee, an order of students walking across the stage, location data associated with the trainee's client device, and / or any other adequate data. As such, the host device can analyze the data to verify the trainee's identity (for example, when the data meets a criteria (s)) and can define a contextual video stream of the trainee, for example, as he or she walks through from the stage to receive a diploma or similar. In other cases, the host device can identify a trainee's family member or friend and can define a contextual video stream of him or her in a similar way.
[0095] Although the modalities have been described above as being performed on specific devices and / or on specific portions of a device, in other modalities, any of the modalities and / or methods described herein can be performed on any suitable device. For example, despite flows of
Petition 870180048519, of 06/07/2018, p. 67/90
64/68 contextual video have been described above as being sent to a host device (for example, the host device 110) for facial recognition and / or image analysis, in other embodiments, any suitable analysis can be performed on or on a device customer. For example, in some cases, a user can capture a video stream (for example, a contextual video stream) through a camera on the client device and in response, the client device can analyze the video to identify any number of registered or similar users in the video stream. In some cases, the analysis can be through a convolutional neural network sent to and / or stored on the client device (for example, stored in memory and associated with the system application). In some cases, the analysis can be pre-processed and / or pre-classified based on, for example, the user's contact list, friends list, established connections, etc., as described above. In some cases, the client device may send a user-specific video stream to any identified user, as described above. In other embodiments, the client device can load and / or send the analyzed video stream and / or the user-specific video stream (s) to the host device 110 and / or the database 140.
[0096] Although video streams and / or image data are described above as being contextual, it should be understood that video stream data and / or image data can be independent of and / or not associated with contextual data. For example, in some cases, a user can capture a video and / or image stream and can upload the video and / or image stream for processing without defining and / or sending contextual data associated with the video stream and / or video data. Image. In some cases, the host device or the like (for example, the host device
Petition 870180048519, of 06/07/2018, p. 68/90
65/68
110) can receive the video stream and / or image data generated by the user and in response, can perform one or more facial recognition processes and / or any other appropriate analytics on the data to define, for example, a video stream user-specific or user-specific image that is independent of contextual data.
[0097] Although the modalities have been specifically shown and described, it will be understood that several changes in form and details can be made. Although various modalities have been described as having specific characteristics and / or combinations of components, other modalities are possible that have a combination of any characteristics and / or components of any of the modalities as discussed above.
[0098] Where methods and / or events described above indicate certain events and / or procedures occurring in a certain order, the ordering of certain events and / or procedures can be modified. In addition, certain events and / or procedures can be executed concurrently in a parallel process when possible, as well as executed sequentially as described above. Although specific facial recognition methods have been described above according to specific modalities, in some cases, any of the facial recognition methods can be combined, augmented, improved, and / or otherwise collectively performed on a data set of facial recognition. For example, in some cases, a facial recognition method may include analyzing facial recognition data using Eigenvectors, Eigenfaces, and / or other 2D analysis, as well as any suitable 3D analysis such as, for example, 3D reconstruction of multiple 2D images. In some cases, the use of a 2D analysis method and a 3D analysis method can, for example, generate more accurate results.
Petition 870180048519, of 06/07/2018, p. 69/90
66/68 accurate with less load on resources (for example, processing devices) that would otherwise result from only a 3D analysis or only a 2D analysis. In some cases, facial recognition can be performed through convolutional neural networks (CNN) and / or through CNN in combination with any two-dimensional (2D) and / or three-dimensional (3D) facial recognition methods. Furthermore, the use of multiple analysis methods can be used, for example, for redundancy, error checking, load balancing, and / or the like. In some cases, the use of multiple analysis methods may allow a system to selectively analyze a set of facial recognition data based at least in part on specific data included in these.
[0099] Some modalities described here refer to a computer storage product with a non-transitory computer-readable medium (can also be referred to as a non-transitory processor-readable medium) that has instructions or computer code on it for perform various computer-implemented operations. The computer-readable medium (or processor-readable medium) is non-transient in the sense that it does not include transient propagation signals per se (for example, an electromagnetic propagation wave that carries information about a transmission medium such as space or cable ). Computer code and medium (may also be referred to as code) can be those designed and built for specific purpose or purposes. Examples of non-transitory computer-readable media include, but are not limited to, magnetic storage media such as hard drives, floppy disks, and magnetic tape; optical storage media such as Compact Disc / Digital Video Discs (CD / DVDs), Compact Disc Read Only Memories (CDROMs), and holographic devices; magnet storage medium
Petition 870180048519, of 06/07/2018, p. 70/90
67/68 optical such as optical discs; carrier wave signal processing modules; and hardware devices that are specially configured to store and execute program code, such as Application Specific Integrated Circuits (ASICs), Programmable Logic Devices (PLDs), Read-Only Memory (ROM) and Random Access Memory (RAM) ). Other embodiments described herein refer to a computer program product, which may include, for example, the instructions and / or computer code discussed here.
[00100] Some modalities and / or methods described here can be executed by software (executed on hardware), hardware, or a combination thereof. Hardware modules can include, for example, a general purpose processor, a field programmable port network (FPGA), and / or an application specific integrated circuit (ASIC). Software modules (run on hardware) can be expressed in a variety of software languages (for example, computer code), including C, C ++, Java ™, Ruby, Visual Basic ™, and / or other object-oriented procedures , or other programming and development language tools. Examples of computer code include, but are not limited to, microcode or microinstructions, machine instructions as produced by a compiler, code used to produce a web service, and files that contain high-level instructions that are executed by a computer using an interpreter. For example, modalities can be implemented using imperative programming languages (for example, C, Fortran, etc.), functional programming languages (Haskell, Erlang, etc.), logical programming languages (for example, Prolog), object-oriented programming (eg Java, C ++, etc.) or other suitable programming languages and / or design tools 870180048519, from 06/07/2018, p. 71/90
68/68 development. Additional examples of computer code include, but are not limited to, control signals, encrypted code, and compressed code.

权利要求:
Claims (20)
[1]
1. Device, characterized by the fact that it comprises:
a memory; and a processor operatively coupled in memory, the processor configured to receive location data from a user device in the first time, the processor configured to store location data in a user profile data structure, the user profile data including a user's face recognition data from the user device associated with the user based on at least one of two-dimensional facial recognition analytics, three-dimensional facial recognition analytics, or convolutional neural networks (CNN), the configured processor to receive, at a second time different from the first time, at least one image from an image capture device, the processor configured to identify a location based at least in part on a set of characteristics within the at least one image received, the processor configured to retrieve a plurality from a database of user profile data structures including the user profile data structure, the processor configured to, for each user profile data structure of the plurality of user profile data structures, compare location data in that user profile data structure with the location, the processor configured to, when the location data of the user profile data structure and the location are within a predetermined distance from each other, determine to be the user associated with the structure User profile data can be identified in at least one image by analyzing at least one image with respect to the user's facial recognition data
Petition 870180048519, of 06/07/2018, p. 73/90
[2]
2/6 based on at least one of the two-dimensional facial recognition analytics, the three-dimensional facial recognition analytics, or CNN to identify a facial recognition confidence score, the processor configured to associate at least one image with the data structure user profile based on the facial recognition confidence score meeting a predetermined criterion.
2. Apparatus according to claim 1, characterized by the fact that the location data is at least one of iBeacon data, Global Positioning Service (GPS) data, a seat identifier, or a Wi- Fi.
[3]
3. Apparatus according to claim 1, characterized by the fact that the user is a first user, the first image capture device is one of a stand-alone camera or a user client device associated with a second user other than the first user.
[4]
4. Apparatus according to claim 1, characterized by the fact that the processor is configured to identify the location by:
perform image processing on at least one image;
identify at least one site landmark based on image processing; and identifying the location by determining a location of the location landmark.
[5]
5. Apparatus according to claim 1, characterized by the fact that the user's face recognition data includes data relating to a user's photograph that is associated with the user profile data structure.
[6]
6. Apparatus according to claim 1, characterized
Petition 870180048519, of 06/07/2018, p. 74/90
3/6 by the fact that at least one image is stored in the database if the user can be identified in at least one image.
[7]
7. Method, characterized by the fact of understanding:
receiving, in a first time, user location data from a user device;
store user location data in a user profile data structure in a database;
receiving, in a second time different from the first time, at least one image from an image capture device;
identify a location based at least in part on a set of features within the at least one image;
compare the user location data of each user profile data structure from a plurality of user profile data structures stored in the database with the location, the user profile data structure being included in the plurality of structures user profile data; and when the user location data of the user profile data structure of the plurality of user profile data structures coincides with the location:
analyze facial recognition data within the user profile data structure for at least one image, and store at least one image as associated with the user profile data structure when at least one image matches the data of facial recognition.
[8]
8. Method according to claim 7, characterized by the fact that user location data is at least one of iBeacon data, Global Positioning Service (GPS) data, a seat number, or a network identifier Wi-Fi.
[9]
9. Method according to claim 7, characterized
Petition 870180048519, of 06/07/2018, p. 75/90
4/6 for the fact of still understanding:
preprocess at least one image to determine contextual information before comparing facial recognition data with at least one image, contextual information including at least one of a location name, a time that at least one image was captured, or a coincident event that occurred when at least one image was captured.
[10]
10. Method according to claim 7, characterized by the fact that it still comprises:
calculate a confidence level for at least one image based on the comparison of facial recognition data with at least one image; and determining that at least one image matches the facial recognition data when the confidence level exceeds a predetermined limit.
[11]
11. Method according to claim 7, characterized in that the at least one image is one of a photograph or a video that includes at least one image, the method further comprising:
dividing the video into a series of images when the at least one image is a video, so that the facial recognition data is compared to each image in the series of images when the facial recognition data is compared to at least one image.
[12]
12. Method according to claim 7, characterized by the fact that it still comprises:
send a signal to graphically render at least one image on the user's device when the at least one image matches the facial recognition data.
Petition 870180048519, of 06/07/2018, p. 76/90
5/6
[13]
13. Method according to claim 7, characterized by the fact that when identifying the location includes identifying the location based at least in part on a background scenario or a background frame included in at least one image.
[14]
14. Method according to claim 7, the method characterized by the fact that it still comprises:
discard at least one image if at least one image does not match the facial recognition data from at least one user profile data structure from the plurality of user profile data structures.
[15]
15. Device, characterized by the fact of understanding:
a memory; and a processor operatively coupled in memory, the processor configured to obtain at least one image, the processor configured to identify a set of characteristics within the at least one image, the processor configured to identify a location in which at least one image was captured based on comparing the feature set with milestone location data stored in a database, the processor configured to select a set of client devices from a plurality of client devices, each client device from the client device set being selected based on user location data associated with that client device being within a predetermined distance from the location, the processor configured to match a client device in the set of client devices with at least one image based on an analysis at least one image and facial recognition data associated with the client device, the processor configured to send a signal so that at least
Petition 870180048519, of 06/07/2018, p. 77/90
6/6 unless an image is rendered on the client device.
[16]
16. Apparatus according to claim 15, characterized by the fact that the analysis is an analysis of facial recognition of at least one image to detect a user associated with facial recognition data, in at least one image.
[17]
17. Apparatus according to claim 15, characterized by the fact that the set of characteristics includes at least one of a site landmark or background scenario.
[18]
18. Apparatus according to claim 15, characterized by the fact that:
the processor is further configured to store at least one image and associate at least one image with the facial recognition data associated with the client device.
[19]
19. Apparatus according to claim 15, characterized by the fact that the client device is matched to at least one image when a confidence level of the at least one image meets a predetermined criterion, the confidence level being determined based on analyzing at least one image and facial recognition data.
[20]
20. Apparatus according to claim 15, characterized in that the at least one image is at least one of a photograph or a video that includes at least one image.

类似技术:

---

公开号 | 公开日 | 专利标题

---

BR112018007979A2|2019-11-12|false positive minimization methods and apparatus in face recognition applications

---

US10900772B2|2021-01-26|Apparatus and methods for facial recognition and video analytics to identify individuals in contextual video streams

---

US9378408B2|2016-06-28|Assisted photo-tagging with facial recognition models

---

US20140337341A1|2014-11-13|Auto-Tagging In Geo-Social Networking System

---

BR112014000615B1|2021-07-13|METHOD TO SELECT VISUAL CONTENT EDITING FUNCTIONS, METHOD TO ADJUST VISUAL CONTENT, AND SYSTEM TO PROVIDE A PLURALITY OF VISUAL CONTENT EDITING FUNCTIONS

---

US9477685B1|2016-10-25|Finding untagged images of a social network member

---

US9122912B1|2015-09-01|Sharing photos in a social network system

---

US20210248354A1|2021-08-12|Methods and apparatus for reducing false positives in facial recognition

---

US11010596B2|2021-05-18|Apparatus and methods for facial recognition systems to identify proximity-based connections

---

WO2021138499A1|2021-07-08|Methods and apparatus for facial recognition on a user device

同族专利:

---

公开号 | 公开日

---

SG10201912947XA|2020-02-27|

---

IL258817D0|2018-06-28|

---

AU2016342028B2|2020-08-20|

---

US20200262931A1|2020-08-20|

---

US10654942B2|2020-05-19|

---

JP2018536929A|2018-12-13|

---

JP6850291B2|2021-03-31|

---

SG11201803263WA|2018-05-30|

---

US20170116466A1|2017-04-27|

---

EP3365838A1|2018-08-29|

---

CN108369652A|2018-08-03|

---

JP2021099852A|2021-07-01|

---

KR20180105636A|2018-09-28|

---

CA3040856A1|2017-04-27|

---

AU2016342028A1|2018-06-07|

---

EP3365838A4|2019-08-28|

---

IL258817A|2021-10-31|

---

WO2017070519A1|2017-04-27|

引用文献:

---

公开号 | 申请日 | 公开日 | 申请人 | 专利标题

---

---

US5144685A|1989-03-31|1992-09-01|Honeywell Inc.|Landmark recognition for autonomous mobile robots|

---

US5694546A|1994-05-31|1997-12-02|Reisman; Richard R.|System for automatic unattended electronic information transport between a server and a client by a vendor provided transport software with a manifest list|

---

US6532345B1|1994-08-02|2003-03-11|L. N. C. J. Limited|Imaging system and method|

---

WO1998010358A1|1996-09-04|1998-03-12|Goldberg David A|Method and system for obtaining person-specific images in a public venue|

---

US20030118216A1|1996-09-04|2003-06-26|Goldberg David A.|Obtaining person-specific images in a public venue|

---

US6819783B2|1996-09-04|2004-11-16|Centerframe, Llc|Obtaining person-specific images in a public venue|

---

US6142876A|1997-08-22|2000-11-07|Cumbers; Blake|Player tracking and identification system|

---

US7330875B1|1999-06-15|2008-02-12|Microsoft Corporation|System and method for recording a presentation for on-demand viewing over a computer network|

---

JP3992909B2|2000-07-03|2007-10-17|富士フイルム株式会社|Personal image providing system|

---

JP3955170B2|2000-07-03|2007-08-08|富士フイルム株式会社|Image search system|

---

US6745186B1|2000-08-17|2004-06-01|Eastman Kodak Company|Product and method for organizing and searching digital images|

---

AU8248301A|2000-08-29|2002-03-13|Imageid Ltd|Indexing, storage and retrieval of digital images|

---

US7376276B2|2000-08-29|2008-05-20|Imageid Ltd|Indexing, storage and retrieval of digital images|

---

US6591068B1|2000-10-16|2003-07-08|Disney Enterprises, Inc|Method and apparatus for automatic image capture|

---

US7321387B2|2001-07-30|2008-01-22|Eastman Kodak Company|System and method for offering imaging services of a customer and participants at a live entertainment event|

---

GB2380556A|2001-10-05|2003-04-09|Hewlett Packard Co|Camera with vocal control and recording|

---

US8321231B2|2001-11-07|2012-11-27|Toshiba Global Commerce Solutions Holding Corporation|Method and apparatus for providing customized souvenir images|

---

GB2386276B|2002-03-09|2004-09-08|Rx Technology Europ Ltd|Image capture and retrieval apparatus|

---

US7636365B2|2004-05-13|2009-12-22|Korea Advanced Institute Of Science And Technology |Smart digital modules and smart digital wall surfaces combining the same, and context aware interactive multimedia system using the same and operation method thereof|

---

JP4789511B2|2004-06-04|2011-10-12|キヤノン株式会社|Status monitoring device and status monitoring system|

---

US7532811B2|2004-07-13|2009-05-12|David Andrew Sauder|System and method for capturing, associating and distributing photographic images|

---

CN100574394C|2004-11-04|2009-12-23|皇家飞利浦电子股份有限公司|For television recommender merges protagonist information|

---

WO2006091869A2|2005-02-25|2006-08-31|Youfinder Intellectual Property Licensing Limited Liability Company|Automated indexing for distributing event photography|

---

US8260674B2|2007-03-27|2012-09-04|David Clifford R|Interactive image activation and distribution system and associate methods|

---

US7966223B2|2005-04-15|2011-06-21|David Clifford R|Interactive image activation and distribution system and associated methods|

---

WO2006113392A2|2005-04-15|2006-10-26|David Clifford R|Interactive image activation and distribution system and associated methods|

---

WO2007011709A2|2005-07-18|2007-01-25|Youfinder Intellectual Property Licensing Limited Liability Company|Manually-assisted automated indexing of images using facial recognition|

---

JP2007052646A|2005-08-18|2007-03-01|Fujifilm Holdings Corp|Image retrieval device, image printer, print ordering system, storefront print terminal device, imaging device, and image retrieval program and method|

---

US7783085B2|2006-05-10|2010-08-24|Aol Inc.|Using relevance feedback in face recognition|

---

US7907755B1|2006-05-10|2011-03-15|Aol Inc.|Detecting facial similarity based on human perception of facial similarity|

---

US7494061B2|2006-06-30|2009-02-24|Evercom Systems, Inc.|Systems and methods for identity verification using continuous biometric monitoring|

---

US7315631B1|2006-08-11|2008-01-01|Fotonation Vision Limited|Real-time face tracking in a digital image acquisition device|

---

US7620218B2|2006-08-11|2009-11-17|Fotonation Ireland Limited|Real-time face tracking with reference images|

---

US7403643B2|2006-08-11|2008-07-22|Fotonation Vision Limited|Real-time face tracking in a digital image acquisition device|

---

US7684651B2|2006-08-23|2010-03-23|Microsoft Corporation|Image-based face search|

---

US8315463B2|2006-11-14|2012-11-20|Eastman Kodak Company|User interface for face recognition|

---

WO2008109172A1|2007-03-07|2008-09-12|Wiklof Christopher A|Recorder with retrospective capture|

---

US8189880B2|2007-05-29|2012-05-29|Microsoft Corporation|Interactive photo annotation based on face clustering|

---

US8014572B2|2007-06-08|2011-09-06|Microsoft Corporation|Face annotation framework with partial clustering and interactive labeling|

---

KR101378372B1|2007-07-12|2014-03-27|삼성전자주식회사|Digital image processing apparatus, method for controlling the same, and recording medium storing program to implement the method|

---

US8144944B2|2007-08-14|2012-03-27|Olympus Corporation|Image sharing system and method|

---

KR101382499B1|2007-10-22|2014-04-21|삼성전자주식회사|Method for tagging video and apparatus for video player using the same|

---

US7391886B1|2008-01-09|2008-06-24|International Business Machines Corporation|Digital camera with image tracking system|

---

US8180112B2|2008-01-21|2012-05-15|Eastman Kodak Company|Enabling persistent recognition of individuals in images|

---

KR101432177B1|2008-01-21|2014-08-22|삼성전자주식회사|Portable device and method for processing the photography the same, and photography processing system having it|

---

US7953690B2|2008-01-25|2011-05-31|Eastman Kodak Company|Discovering social relationships from personal photo collections|

---

EP2651100A1|2008-02-22|2013-10-16|Security First Corporation|Systems and methods for secure workgroup management and communication|

---

US8666198B2|2008-03-20|2014-03-04|Facebook, Inc.|Relationship mapping employing multi-dimensional context including facial recognition|

---

EP2618290A3|2008-04-02|2014-08-06|Google, Inc.|Method and apparatus to incorporate automatic face recognition in digital image collections|

---

US8406531B2|2008-05-15|2013-03-26|Yahoo! Inc.|Data access based on content of image recorded by a mobile device|

---

US8284990B2|2008-05-21|2012-10-09|Honeywell International Inc.|Social network construction based on data association|

---

US9646025B2|2008-05-27|2017-05-09|Qualcomm Incorporated|Method and apparatus for aggregating and presenting data associated with geographic locations|

---

US20090316961A1|2008-06-21|2009-12-24|Microsoft Corporation|Method for tagging image content|

---

JP2010009389A|2008-06-27|2010-01-14|Toshiba Corp|Dictionary information registration device, dictionary information registration method, face authentication device, and access control system|

---

JP5049899B2|2008-06-30|2012-10-17|キヤノン株式会社|Imaging apparatus and control method thereof|

---

WO2010006367A1|2008-07-16|2010-01-21|Imprezzeo Pty Ltd|Facial image recognition and retrieval|

---

US20100036875A1|2008-08-07|2010-02-11|Honeywell International Inc.| system for automatic social network construction from image data|

---

US8204437B1|2008-08-08|2012-06-19|Srr Patent Holdings, Llc|Wireless image distribution system and method|

---

US8886206B2|2009-05-01|2014-11-11|Digimarc Corporation|Methods and systems for content processing|

---

EP2321953B1|2008-09-02|2013-11-06|Ecole Polytechnique Federale De Lausanne |Image annotation on portable devices|

---

US8457366B2|2008-12-12|2013-06-04|At&T Intellectual Property I, L.P.|System and method for matching faces|

---

US8442922B2|2008-12-24|2013-05-14|Strands, Inc.|Sporting event image capture, processing and publication|

---

US8254699B1|2009-02-02|2012-08-28|Google Inc.|Automatic large scale video object recognition|

---

US8782709B2|2009-02-19|2014-07-15|Hulu, LLC|Method and apparatus for providing a program guide having search parameter aware thumbnails|

---

US8392957B2|2009-05-01|2013-03-05|T-Mobile Usa, Inc.|Automatic content tagging, such as tagging digital images via a wireless cellular network using metadata and facial recognition|

---

US8325999B2|2009-06-08|2012-12-04|Microsoft Corporation|Assisted face recognition tagging|

---

US20110022529A1|2009-07-22|2011-01-27|Fernando Barsoba|Social network creation using image recognition|

---

US8670597B2|2009-08-07|2014-03-11|Google Inc.|Facial recognition with social network aiding|

---

US8510383B2|2009-09-14|2013-08-13|Clixtr, Inc.|Method for providing event based media streams|

---

US20110066431A1|2009-09-15|2011-03-17|Mediatek Inc.|Hand-held input apparatus and input method for inputting data to a remote receiving device|

---

US8438484B2|2009-11-06|2013-05-07|Sony Corporation|Video preview module to enhance online video experience|

---

US8386620B2|2009-12-15|2013-02-26|Apple Inc.|Ad hoc networking based on content and location|

---

US8416997B2|2010-01-27|2013-04-09|Apple Inc.|Method of person identification using social connections|

---

US20110188742A1|2010-02-02|2011-08-04|Jie Yu|Recommending user image to social network groups|

---

US9465993B2|2010-03-01|2016-10-11|Microsoft Technology Licensing, Llc|Ranking clusters based on facial image analysis|

---

US20110211737A1|2010-03-01|2011-09-01|Microsoft Corporation|Event Matching in Social Networks|

---

US8957981B2|2010-03-03|2015-02-17|Intellectual Ventures Fund 83 Llc|Imaging device for capturing self-portrait images|

---

WO2011119859A2|2010-03-24|2011-09-29|Hameed Khan|Proximity-based social networking|

---

US20110257985A1|2010-04-14|2011-10-20|Boris Goldstein|Method and System for Facial Recognition Applications including Avatar Support|

---

US20110307399A1|2010-06-09|2011-12-15|Brian Holmes|Live Event Social Networking System|

---

JP5545084B2|2010-07-08|2014-07-09|ソニー株式会社|Information processing apparatus, information processing method, and program|

---

US8270684B2|2010-07-27|2012-09-18|Google Inc.|Automatic media sharing via shutter click|

---

US9202091B2|2010-09-02|2015-12-01|Intelleflex Corporation|RFID reader with camera, video, and/or audio capture device|

---

US8928760B2|2010-12-07|2015-01-06|Verizon Patent And Licensing Inc.|Receiving content and approving content for transmission|

---

US8341145B2|2010-12-20|2012-12-25|Microsoft Corporation|Face recognition using social data|

---

US9135500B2|2011-02-18|2015-09-15|Google Inc.|Facial recognition|

---

JP5530572B2|2011-03-07|2014-06-25|ケービーエー２，インコーポレイティド|System and method for analytical data collection from an image provider at an event or geographic location|

---

US9317530B2|2011-03-29|2016-04-19|Facebook, Inc.|Face recognition based on spatial and temporal proximity|

---

US20120265758A1|2011-04-14|2012-10-18|Edward Han|System and method for gathering, filtering, and displaying content captured at an event|

---

US8631084B2|2011-04-29|2014-01-14|Facebook, Inc.|Dynamic tagging recommendation|

---

US8918463B2|2011-04-29|2014-12-23|Facebook, Inc.|Automated event tagging|

---

US8971924B2|2011-05-23|2015-03-03|Apple Inc.|Identifying and locating users on a mobile network|

---

US9082235B2|2011-07-12|2015-07-14|Microsoft Technology Licensing, Llc|Using facial data for device authentication or subject identification|

---

US10089327B2|2011-08-18|2018-10-02|Qualcomm Incorporated|Smart camera for sharing pictures automatically|

---

US8885960B2|2011-10-05|2014-11-11|Microsoft Corporation|Linking photographs via face, time, and location|

---

US9131147B2|2011-10-07|2015-09-08|Fuji Xerox Co., Ltd.|System and method for detecting and acting on multiple people crowding a small display for information sharing|

---

US9143601B2|2011-11-09|2015-09-22|Microsoft Technology Licensing, Llc|Event-based media grouping, playback, and sharing|

---

US9087273B2|2011-11-15|2015-07-21|Facebook, Inc.|Facial recognition using social networking information|

---

US8723962B2|2011-12-13|2014-05-13|International Business Machines Corporation|Camera based information exchange mechanism|

---

WO2013149357A1|2012-04-01|2013-10-10|Intel Corporation|Analyzing human gestural commands|

---

US9407860B2|2012-04-06|2016-08-02|Melvin Lee Barnes, JR.|System, method and computer program product for processing image data|

---

US9137246B2|2012-04-09|2015-09-15|Brivas Llc|Systems, methods and apparatus for multivariate authentication|

---

US9179021B2|2012-04-25|2015-11-03|Microsoft Technology Licensing, Llc|Proximity and connection based photo sharing|

---

US8849041B2|2012-06-04|2014-09-30|Comcast Cable Communications, Llc|Data recognition in content|

---

US8710751B2|2012-07-26|2014-04-29|Polestar Electric Industries Co., Ltd.|LED string with a capability to maintain a current path and LED unit thereof|

---

US9204247B2|2012-08-13|2015-12-01|Google Inc.|Proximity-based notifications for photos|

---

US8560625B1|2012-09-01|2013-10-15|Google Inc.|Facilitating photo sharing|

---

US9141848B2|2012-09-04|2015-09-22|Intel Corporation|Automatic media distribution|

---

AU2012101375A4|2012-09-06|2012-10-18|Oracle Recording's|FutureNetID|

---

US10027727B1|2012-11-21|2018-07-17|Ozog Media, LLC|Facial recognition device, apparatus, and method|

---

US10027726B1|2012-11-21|2018-07-17|Ozog Media, LLC|Device, apparatus, and method for facial recognition|

---

US10019136B1|2012-11-21|2018-07-10|Ozog Media, LLC|Image sharing device, apparatus, and method|

---

KR20150100795A|2012-12-19|2015-09-02|팬픽스, 엘엘씨|Image capture, processing and delivery at group events|

---

US9712800B2|2012-12-20|2017-07-18|Google Inc.|Automatic identification of a notable moment|

---

US20140267618A1|2013-03-15|2014-09-18|Google Inc.|Capturing and Refocusing Imagery|

---

US20140342330A1|2013-05-17|2014-11-20|Gary A. Freeman|Cameras for Emergency Rescue|

---

US20140350840A1|2013-05-23|2014-11-27|Cellco Partnership D/B/A Verizon Wireless|Crowd proximity device|

---

CN105190699B|2013-06-05|2019-09-03|英特尔公司|Karaoke incarnation animation based on facial motion data|

---

US9104907B2|2013-07-17|2015-08-11|Emotient, Inc.|Head-pose invariant recognition of facial expressions|

---

WO2015031863A1|2013-08-29|2015-03-05|FanPix, LLC|Imaging attendees at event venues|

---

US20150062334A1|2013-08-29|2015-03-05|FanPix, LLC|Systems and Methods for Image Capture and Distribution in Venues|

---

US20150066920A1|2013-09-04|2015-03-05|Google Inc.|Media clip sharing on social networks|

---

US9807161B2|2013-09-16|2017-10-31|Axis Ab|Distributed events in an access control system|

---

US10460151B2|2013-09-17|2019-10-29|Cloudspotter Technologies, Inc.|Private photo sharing system, method and network|

---

JP5935777B2|2013-09-17|2016-06-15|コニカミノルタ株式会社|Image processing apparatus, image processing method, image processing system, and program|

---

US9569656B2|2013-12-06|2017-02-14|Google Inc.|Local real-time facial recognition|

---

US9773162B2|2013-12-12|2017-09-26|Evernote Corporation|User discovery via digital ID and face recognition|

---

US20150172787A1|2013-12-13|2015-06-18|Amazon Technologies, Inc.|Customized movie trailers|

---

US9007420B1|2014-01-10|2015-04-14|Securus Technologies, Inc.|Verifying presence of authorized persons during an electronic visitation|

---

US20150227780A1|2014-02-13|2015-08-13|FacialNetwork, Inc.|Method and apparatus for determining identity and programing based on image features|

---

US9189682B2|2014-02-13|2015-11-17|Apple Inc.|Systems and methods for sending digital images|

---

US10121060B2|2014-02-13|2018-11-06|Oath Inc.|Automatic group formation and group detection through media recognition|

---

US9858601B2|2014-03-05|2018-01-02|Ricoh Co., Ltd.|Grouping users based on user profiles|

---

US9852364B2|2014-03-19|2017-12-26|Hulu, LLC|Face track recognition with multi-sample multi-view weighting|

---

US20150304436A1|2014-04-16|2015-10-22|Facebook, Inc.|Nearby Friend Notifications on Online Social Networks|

---

US9396354B1|2014-05-28|2016-07-19|Snapchat, Inc.|Apparatus and method for automated privacy protection in distributed images|

---

US9420315B2|2014-06-17|2016-08-16|Bruce Melanson|Systems and methods for sharing video data via social media|

---

US10264175B2|2014-09-09|2019-04-16|ProSports Technologies, LLC|Facial recognition for event venue cameras|

---

US20160071101A1|2014-09-09|2016-03-10|Tyson York Winarski|Selfie financial security transaction system|

---

US9516472B2|2014-10-10|2016-12-06|Verizon Patent And Licensing Inc.|Method and system for evaluating a user response to a presence based action|

---

US20160150124A1|2014-11-24|2016-05-26|Kyocera Document Solutions Inc.|Image Forming Apparatus with User Identification Capabilities|

---

US20160191434A1|2014-12-24|2016-06-30|Blue Yonder Labs Llc|System and method for improved capture, storage, search, selection and delivery of images across a communications network|

---

US10094655B2|2015-07-15|2018-10-09|15 Seconds of Fame, Inc.|Apparatus and methods for facial recognition and video analytics to identify individuals in contextual video streams|

---

JP6850291B2|2015-10-21|2021-03-31|１５ セカンズ オブ フェイム，インコーポレイテッド|Methods and devices for minimizing false positives in face recognition applications|

---

US9668002B1|2015-11-19|2017-05-30|Amazon Technologies, Inc.|Identification of live streaming content|

---

KR102233175B1|2017-01-05|2021-03-29|한국전자통신연구원|Method for determining signature actor and for identifying image based on probability of appearance of signature actor and apparatus for the same|

---

US10572767B2|2017-04-12|2020-02-25|Netflix, Inc.|Scene and shot detection and characterization|

---

US10885396B2|2017-05-24|2021-01-05|Amazon Technologies, Inc.|Generating composite images using audio/video recording and communication devices|

---

KR20190069920A|2017-12-12|2019-06-20|한국전자통신연구원|Apparatus and method for recognizing character in video contents|JP6504453B2|2015-07-01|2019-04-24|カシオ計算機株式会社|Image transmitting apparatus, image transmitting method and program|

---

US10094655B2|2015-07-15|2018-10-09|15 Seconds of Fame, Inc.|Apparatus and methods for facial recognition and video analytics to identify individuals in contextual video streams|

---

CN105095873B|2015-07-31|2018-12-18|小米科技有限责任公司|Photo be shared method, apparatus|

---

JP6850291B2|2015-10-21|2021-03-31|１５ セカンズ オブ フェイム，インコーポレイテッド|Methods and devices for minimizing false positives in face recognition applications|

---

CN106096009A|2016-06-23|2016-11-09|北京小米移动软件有限公司|Method for generating message and device|

---

US10140675B2|2016-11-28|2018-11-27|Google Llc|Image grid with selectively prominent images|

---

CN108876944A|2017-05-12|2018-11-23|阿里巴巴集团控股有限公司|A kind of method and apparatus of vehicle-mounted payment|

---

WO2019083508A1|2017-10-24|2019-05-02|Hewlett-Packard Development Company, L.P.|Facial recognitions based on contextual information|

---

KR20190069920A|2017-12-12|2019-06-20|한국전자통신연구원|Apparatus and method for recognizing character in video contents|

---

JP6975666B2|2018-03-20|2021-12-01|株式会社日立ソリューションズ・クリエイト|Person identification system and person identification method|

---

US10936856B2|2018-08-31|2021-03-02|15 Seconds of Fame, Inc.|Methods and apparatus for reducing false positives in facial recognition|

---

US11010596B2|2019-03-07|2021-05-18|15 Seconds of Fame, Inc.|Apparatus and methods for facial recognition systems to identify proximity-based connections|

---

US20200401791A1|2019-06-19|2020-12-24|Samsung Electronics Company, Ltd.|Methods and Systems for Dynamic and Incremental Face Recognition|

---

EP3855348A1|2020-01-27|2021-07-28|Microsoft Technology Licensing, LLC|Error management|

---

WO2021163075A1|2020-02-13|2021-08-19|Best Apps, Llc|Computer aided systems and methods for creating custom products|

法律状态:
2020-07-14| B06U| Preliminary requirement: requests with searches performed by other patent offices: procedure suspended [chapter 6.21 patent gazette]|

---

2020-11-24| B11B| Dismissal acc. art. 36, par 1 of ipl - no reply within 90 days to fullfil the necessary requirements|

---

2021-10-13| B350| Update of information on the portal [chapter 15.35 patent gazette]|

优先权:

---

申请号 | 申请日 | 专利标题

---

US201562244419P| true| 2015-10-21|2015-10-21|

---

PCT/US2016/058189|WO2017070519A1|2015-10-21|2016-10-21|Methods and apparatus for false positive minimization in facial recognition applications|

---